nesquena/hermes-webui

Test isolation: test_title_aux_routing.py leaks stub agent.auxiliary_client into context_compressor import

Closed

#6,630 opened on Jul 30, 2026

 (0 comments) (0 reactions) (0 assignees)Python (2,386 forks)github user discovery
bugcleanuphelp wantedsprint-candidate

Repository metrics

Stars
 (17,368 stars)
PR merge metrics
 (Avg merge 14h 31m) (314 merged PRs in 30d)

Description

Test-isolation bug: test_title_aux_routing.py leaks a stub agent.auxiliary_client into context_compressor import

Symptom

When tests/test_title_aux_routing.py runs before tests/test_issue4685_post_compression_context_metering.py in the same process, the metering test fails at collection/setup:

ImportError: cannot import name 'call_llm' from 'agent.auxiliary_client' (unknown location)
agent/context_compressor.py:28: ImportError

test_issue4685_... does pytest.importorskip("agent.context_compressor"), and context_compressor.py:28 does from agent.auxiliary_client import call_llm.

Reproduction (deterministic, on current origin/master)

python -m pytest tests/test_title_aux_routing.py \
                 tests/test_issue4685_post_compression_context_metering.py -p no:xdist -q
# → 1 failed (test_post_compression_estimate_uses_compressor_budget_counter_without_metadata_estimators)

Each file passes cleanly in isolation (8/8 and 66/66 respectively). The failure only appears in the adjacency, so it stays hidden under most shard orderings and surfaces intermittently in full-suite runs whenever collection order places these two files together.

Root cause

tests/test_title_aux_routing.py installs a stub module at import time:

_aux_stub = types.ModuleType('agent.auxiliary_client')
sys.modules.setdefault('agent', _agent_stub)
sys.modules.setdefault('agent.auxiliary_client', _aux_stub)   # ← leaks process-wide

The stub has no call_llm. setdefault only installs it if agent.auxiliary_client isn't already imported, but once installed it persists in sys.modules for the rest of the process. When context_compressor is later imported for the first time (via importorskip), its top-level from agent.auxiliary_client import call_llm resolves against the stub and raises ImportError.

Fix shape (options)

  1. Scope the stub to the test module with an autouse fixture that saves/restores sys.modules['agent.auxiliary_client'] (and 'agent') around the test file, rather than a module-import-time setdefault that never gets torn down. (preferred — contains the blast radius)
  2. Or give the stub a call_llm attribute (and the other names context_compressor imports) so a downstream real import doesn't fault. (weaker — still leaks a fake module)
  3. Or make context_compressor's import of call_llm lazy/guarded. (touches production for a test problem — least preferred)

Impact

No production impact — the two production modules are correct in isolation. This is a suite-hygiene defect that produces a confusing phantom failure in full-suite gate runs and can mask/confuse real regressions during PR gating (observed while gating an unrelated title-routing PR).

Contributor guide