vllm-project/semantic-router
feature: add common per-attempt Looper traces and metrics
開放
#2,855 建立於 2026年8月11日
area/corearea/momarea/observabilityenhancementhelp wantedpriority/P1roadmap
倉庫指標
- 星標
- (4,293 顆星)
- PR 合併指標
- (PR 指標待抓取)
描述
Summary
Define a common per-attempt execution trace for Looper-family algorithms and persist bounded attempt data to Router Replay and metrics.
Parent: #2336 Depends on selected-model correctness: #2854 Related response-contract work: #2200, #2205
Motivation
The shared model response already records per-call usage and latency, but algorithm observability is uneven:
- confidence scores and acceptance decisions are primarily logs;
- Fusion exposes algorithm-specific panel and grounding details;
- ReMoM and Workflows expose different intermediate structures;
- aggregate debug headers cannot explain verifier regret, early stopping, failures, or wasted work.
Operators need one bounded contract that distinguishes attempts from the final selected result.
Proposed contract
Each attempt should be able to record:
- stable stage/attempt identity and algorithm;
- model and role;
- prompt, completion, and total token usage;
- upstream latency;
- optional verifier method and score;
- accepted, selected, cancelled, timed-out, or failed state;
- bounded stop/failure reason without prompt or response content.
Scope
- Add a shared attempt-trace type under
pkg/looper. - Populate confidence first, then adapt Fusion, ReMoM, and Workflows without removing their richer algorithm-specific traces.
- Persist attempts in Router Replay and expose low-cardinality aggregate metrics.
- Keep public response headers limited to final facts; detailed attempts remain replay/debug data.
- Define size/cardinality limits for serialized attempts.
Non-goals
- Do not store full prompts, reasoning traces, tool arguments, credentials, or response bodies in metrics.
- Do not redesign the entire replay record.
- Do not define compute-budget enforcement in this issue.
Acceptance criteria
- Confidence replay shows every attempted model, score, threshold result, usage, and latency.
- Fusion/ReMoM/Workflows can emit the same common attempt envelope alongside existing traces.
- The final selected model is represented separately from attempted models.
- Cancellation, timeout, and partial failure have stable reason codes.
- Trace size and metric labels are bounded and tested.
- No sensitive prompt or credential content is emitted.
Validation
- Unit tests for trace serialization and bounds.
- Algorithm-specific trace tests.
- Router Replay persistence tests.
- Metrics contract tests and affected E2E coverage.