vllm-project/semantic-router

refactor: unify Looper candidate-panel execution semantics

开放

#2,856 创建于 2026年8月11日

 (1 条评论) (0 个反应) (0 位负责人)Go (699 个派生)github user discovery
area/corearea/momenhancementhelp wantedpriority/P1roadmap

仓库指标

星标
 (4,293 个星标)
PR 合并指标
 (PR 指标待抓取)

描述

Summary

Extract a shared candidate-panel execution primitive for Looper-family algorithms so concurrency, ordering, timeout, quorum, cancellation, draining, accounting, and partial-failure behavior have one contract.

Parent: #2336 Related amplification budget: #1456 Related attempt trace: #2855

Current behavior

Fusion, ReMoM, Workflows, and Ratings each own a similar but different goroutine/semaphore/collector implementation:

  • Fusion and Workflows actively cancel from their collectors when quorum is reached.
  • ReMoM reaches quorum in its collector and cancels through the outer execution context on return.
  • Ratings waits for every candidate and has no quorum or round timeout.
  • Ordering, error policy, cancellation accounting, and late-result handling are algorithm-specific.

The duplication makes resource accounting and future Best-of-N/adaptive sampling harder to implement safely.

Scope

  • Introduce a shared internal panel executor with explicit inputs for concurrency, timeout/deadline, quorum, ordering, and error policy.
  • Define cancellation ownership and whether/how outstanding results are drained and accounted.
  • Preserve deterministic candidate ordering independent of completion order.
  • Return structured attempted/successful/failed/cancelled results suitable for #2855 and #1456.
  • Migrate Fusion and ReMoM first, then Workflows and Ratings with behavior-preserving tests.

Non-goals

  • Do not add Best-of-N, adaptive-N, semantic deduplication, or a new public algorithm.
  • Do not change algorithm output bodies during the initial migration.
  • Do not merge tool-trajectory state into the generic executor.

Acceptance criteria

  • All migrated algorithms share one tested concurrency/quorum/cancellation primitive.
  • Existing ordering and partial-failure behavior remain compatible unless an explicit bug is documented.
  • Exact-quorum, over-quorum, timeout, fail-fast, skip, cancellation, and all-failed cases are tested.
  • No goroutine or semaphore-slot leaks occur under cancellation.
  • Attempt and usage accounting has a documented contract for cancelled/in-flight calls.
  • Existing Looper microbenchmarks do not regress beyond repository thresholds.

Validation

  • Targeted Looper unit tests under -race.
  • Existing Fusion/ReMoM/Workflows/Ratings tests.
  • Looper performance benchmark gate.

贡献者指南