vllm-project/semantic-router

feature: add common per-attempt Looper traces and metrics

开放

#2,855 创建于 2026年8月11日

 (1 条评论) (0 个反应) (0 位负责人)Go (699 个派生)github user discovery
area/corearea/momarea/observabilityenhancementhelp wantedpriority/P1roadmap

仓库指标

星标
 (4,293 个星标)
PR 合并指标
 (PR 指标待抓取)

描述

Summary

Define a common per-attempt execution trace for Looper-family algorithms and persist bounded attempt data to Router Replay and metrics.

Parent: #2336 Depends on selected-model correctness: #2854 Related response-contract work: #2200, #2205

Motivation

The shared model response already records per-call usage and latency, but algorithm observability is uneven:

  • confidence scores and acceptance decisions are primarily logs;
  • Fusion exposes algorithm-specific panel and grounding details;
  • ReMoM and Workflows expose different intermediate structures;
  • aggregate debug headers cannot explain verifier regret, early stopping, failures, or wasted work.

Operators need one bounded contract that distinguishes attempts from the final selected result.

Proposed contract

Each attempt should be able to record:

  • stable stage/attempt identity and algorithm;
  • model and role;
  • prompt, completion, and total token usage;
  • upstream latency;
  • optional verifier method and score;
  • accepted, selected, cancelled, timed-out, or failed state;
  • bounded stop/failure reason without prompt or response content.

Scope

  • Add a shared attempt-trace type under pkg/looper.
  • Populate confidence first, then adapt Fusion, ReMoM, and Workflows without removing their richer algorithm-specific traces.
  • Persist attempts in Router Replay and expose low-cardinality aggregate metrics.
  • Keep public response headers limited to final facts; detailed attempts remain replay/debug data.
  • Define size/cardinality limits for serialized attempts.

Non-goals

  • Do not store full prompts, reasoning traces, tool arguments, credentials, or response bodies in metrics.
  • Do not redesign the entire replay record.
  • Do not define compute-budget enforcement in this issue.

Acceptance criteria

  • Confidence replay shows every attempted model, score, threshold result, usage, and latency.
  • Fusion/ReMoM/Workflows can emit the same common attempt envelope alongside existing traces.
  • The final selected model is represented separately from attempted models.
  • Cancellation, timeout, and partial failure have stable reason codes.
  • Trace size and metric labels are bounded and tested.
  • No sensitive prompt or credential content is emitted.

Validation

  • Unit tests for trace serialization and bounds.
  • Algorithm-specific trace tests.
  • Router Replay persistence tests.
  • Metrics contract tests and affected E2E coverage.

贡献者指南