refactor(gapic-generator): harmonize and deduplicate showcase nox test sessions

Open
  • Is something wrong?
#18,527 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
55/100
Issue type
Refactor
Clarity
Mostly clear
Activity status
Active
Tech stack
python
Domain
build-system, testing-qa

Research direction

Read packages/gapic-generator/noxfile.py, focusing on showcase_library, run_showcase_unit_tests, and the showcase sessions around lines 397–629. Run the existing Showcase nox sessions before changing them, then verify that unified system and unit runners preserve each listed variant, options, paths, and pytest behavior.

Written by the indexing model from the issue text.

Description

Problem Summary

In packages/gapic-generator/noxfile.py, the test suite for the Showcase client has grown into over 10 distinct showcase_* sessions across system tests and unit tests.

While proto compilation and client generation were previously consolidated into the with showcase_library(...) context manager, the test execution logic inside the context blocks has suffered from copy-paste sprawl:

  1. System Tests Duplication:
    The sessions showcase, showcase_w_rest_async, showcase_mtls, and showcase_pqc (lines 397–535) each duplicate a ~25-line block of setup, dependency installation, path resolution, and py.test execution. There is no shared run_showcase_system_tests() helper. Adding a flag, environment override (such as local core installation), or pytest option requires touching 4 separate sessions.

  2. Unit Tests Session Explosion:
    There are 6 distinct unit test sessions (showcase_unit, showcase_unit_w_rest_async, showcase_unit_alternative_templates, showcase_unit_add_iam_methods, showcase_unit_mixins, showcase_unit_alternative_templates_mixins across lines 566–629) that simply pass different template options or flags to showcase_library and then call run_showcase_unit_tests().

This proliferation leads to maintenance drift, inconsistencies in flags, and makes targeted testing harder.

Proposed Refactor

Consolidate the duplicated sessions into unified, maintainable runners using @nox.parametrize or shared helper functions.

1. System Tests Refactor (Consolidate 4+ sessions into 1)

Instead of 4 copy-pasted sessions:

# Before: 4 duplicate sessions (showcase, showcase_w_rest_async, showcase_mtls, showcase_pqc)
@nox.session(python=ALL_PYTHON)
def showcase(session):
    with showcase_library(session):
        session.install("pytest", "pytest-asyncio<1.0.0")
        ...
        session.run("py.test", ...)

@nox.session(python=ALL_PYTHON)
def showcase_w_rest_async(session):
    with showcase_library(session, rest_async_io_enabled=True):
        session.install("pytest", "pytest-asyncio<1.0.0")
        ...
        session.run("py.test", ...)

Consolidate into a single parameterized session with a shared runner:

# After: Single parameterized system test session
SYSTEM_VARIANTS = {
    "grpc": {"rest_async": False, "extra_args": []},
    "rest_async": {"rest_async": True, "extra_args": []},
    "mtls": {"rest_async": False, "extra_args": ["--mtls"]},
    "pqc": {"rest_async": False, "extra_args": ["--tls"], "test_path": "tests/system/test_pqc.py"},
}

@nox.session(python=ALL_PYTHON)
@nox.parametrize("variant", list(SYSTEM_VARIANTS.keys()))
def showcase_system(session, variant):
    """Run all Showcase system tests through a unified runner."""
    config = SYSTEM_VARIANTS[variant]
    with showcase_library(session, rest_async_io_enabled=config["rest_async"]):
        session.install("pytest", "pytest-asyncio<1.0.0")
        if variant == "pqc":
            session.install("--upgrade", "grpcio>=1.83.0", "grpcio-status>=1.83.0")
        
        test_path = config.get("test_path", "tests/system")
        session.run("py.test", "--quiet", *config["extra_args"], *(session.posargs or [test_path]))
2. Unit Tests Refactor (Consolidate 6 sessions into 1)

Instead of 6 distinct sessions calling run_showcase_unit_tests(), parametrize the template and option configurations:

# After: Single parameterized unit test session
UNIT_CONFIGS = {
    "default": {"templates": "DEFAULT", "other_opts": ()},
    "rest_async": {"templates": "DEFAULT", "rest_async_io_enabled": True},
    "ads_old_naming": {"templates": ADS_TEMPLATES, "other_opts": ("old-naming",)},
    "add_iam_methods": {"other_opts": ("add-iam-methods",), "fail_under": 100},
    "mixins": {"include_service_yaml": True},
    "ads_mixins": {"templates": ADS_TEMPLATES, "other_opts": ("old-naming",), "include_service_yaml": True},
}

@nox.session(python=ALL_PYTHON)
@nox.parametrize("config_name", list(UNIT_CONFIGS.keys()))
def showcase_unit(session, config_name):
    """Run all Showcase unit tests across template configurations."""
    cfg = UNIT_CONFIGS[config_name]
    with showcase_library(
        session,
        templates=cfg.get("templates", "DEFAULT"),
        other_opts=cfg.get("other_opts", ()),
        rest_async_io_enabled=cfg.get("rest_async_io_enabled", False),
        include_service_yaml=cfg.get("include_service_yaml", False),
    ) as lib:
        session.chdir(lib)
        run_showcase_unit_tests(session, fail_under=cfg.get("fail_under", 100))
Benefits
  1. Zero Drift: All test invocation logic, constraint handling, and pytest argument handling exist in single shared locations.
  2. Simplified CI Workflows: Matrix jobs in GitHub Actions can cleanly target showcase_system or showcase_unit with -k <variant>.
  3. Easier Extensibility: Adding canaries, local dependency overrides, or new transport profiles requires modifying a config dictionary rather than copy-pasting an entire session.
Dominant language
Python
Stars
5.4k
Forks
1.8k
Avg merge
2d 7h
Merged PRs (30d)
173

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from googleapis/google-cloud-python

All issues in googleapis/google-cloud-python

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.