vllm-project/semantic-router

test: expand Looper E2E coverage beyond confidence headers

Ouverte

#2 859 ouverte le 11 août 2026

 (1 commentaire) (0 réaction) (0 personne assignée)Go (699 forks)github user discovery
area/momarea/test-and-releasearea/testinge2eenhancementhelp wantedpriority/P1roadmap

Métriques du dépôt

Stars
 (4 293 étoiles)
Métriques de merge PR
 (Métriques PR en attente)

Description

Summary

Expand end-to-end coverage for production Looper algorithms beyond the current confidence-focused aggregate header probe.

Parent: #2336 Related common trace contract: #2855 Related candidate executor: #2856

Motivation

Unit coverage exists for Fusion, ReMoM, Workflows, grounding, tool state, and output contracts, but the E2E surface does not currently prove the corresponding ExtProc loopback, ImmediateResponse, replay/debug, and response-shaping contracts together.

Behavior-visible routing and API changes need end-to-end protection before Looper execution is refactored or extended.

Scope

Add focused E2E cases for:

  • Fusion quorum, partial panel failure, final synthesis, and selected-model attribution;
  • ReMoM multi-round execution, final synthesis, and aggregated usage;
  • Workflows planner/static execution and one pause/resume tool round trip;
  • output-contract normalization and fallback behavior;
  • timeout/cancellation and typed all-failed responses;
  • debug/replay visibility without restoring detailed Looper headers to the default public surface.

Tests should use bounded deterministic fixtures and avoid requiring proprietary providers.

Non-goals

  • Do not turn E2E into a quality benchmark.
  • Do not duplicate every unit-test permutation.
  • Do not rely on private infrastructure, credentials, or model endpoints.

Acceptance criteria

  • Each production Looper family has at least one successful E2E path.
  • Fusion partial success and all-failed behavior are covered.
  • ReMoM proves more than one round and correct aggregate usage.
  • Workflows proves tool pause/resume across requests.
  • Output-contract fallback is exercised through the public API.
  • Selected model, algorithm, iteration count, usage, and replay/debug facts are consistent.
  • Cases are registered in the affected E2E profile map and remain bounded in CI.

Validation

  • Run the affected E2E profile selected by the repository harness.
  • Run targeted pkg/looper and pkg/extproc tests for fixture parity.

Guide contributeur