vllm-project/semantic-router
refactor: unify Looper candidate-panel execution semantics
Open
#2,856 opened on Aug 11, 2026
area/corearea/momenhancementhelp wantedpriority/P1roadmap
Repository metrics
- Stars
- (4,293 stars)
- PR merge metrics
- (PR metrics pending)
Description
Summary
Extract a shared candidate-panel execution primitive for Looper-family algorithms so concurrency, ordering, timeout, quorum, cancellation, draining, accounting, and partial-failure behavior have one contract.
Parent: #2336 Related amplification budget: #1456 Related attempt trace: #2855
Current behavior
Fusion, ReMoM, Workflows, and Ratings each own a similar but different goroutine/semaphore/collector implementation:
- Fusion and Workflows actively cancel from their collectors when quorum is reached.
- ReMoM reaches quorum in its collector and cancels through the outer execution context on return.
- Ratings waits for every candidate and has no quorum or round timeout.
- Ordering, error policy, cancellation accounting, and late-result handling are algorithm-specific.
The duplication makes resource accounting and future Best-of-N/adaptive sampling harder to implement safely.
Scope
- Introduce a shared internal panel executor with explicit inputs for concurrency, timeout/deadline, quorum, ordering, and error policy.
- Define cancellation ownership and whether/how outstanding results are drained and accounted.
- Preserve deterministic candidate ordering independent of completion order.
- Return structured attempted/successful/failed/cancelled results suitable for #2855 and #1456.
- Migrate Fusion and ReMoM first, then Workflows and Ratings with behavior-preserving tests.
Non-goals
- Do not add Best-of-N, adaptive-N, semantic deduplication, or a new public algorithm.
- Do not change algorithm output bodies during the initial migration.
- Do not merge tool-trajectory state into the generic executor.
Acceptance criteria
- All migrated algorithms share one tested concurrency/quorum/cancellation primitive.
- Existing ordering and partial-failure behavior remain compatible unless an explicit bug is documented.
- Exact-quorum, over-quorum, timeout, fail-fast, skip, cancellation, and all-failed cases are tested.
- No goroutine or semaphore-slot leaks occur under cancellation.
- Attempt and usage accounting has a documented contract for cancelled/in-flight calls.
- Existing Looper microbenchmarks do not regress beyond repository thresholds.
Validation
- Targeted Looper unit tests under
-race. - Existing Fusion/ReMoM/Workflows/Ratings tests.
- Looper performance benchmark gate.