vllm-project/semantic-router

feature: make Fusion adaptive and support repeated self-sampling

Offen

#2.865 geöffnet am 11.08.2026

 (0 Kommentare) (0 Reaktionen) (0 zugewiesene Personen)Go (699 Forks)github user discovery
area/corearea/momarea/researchenhancementhelp wantedpriority/P2roadmap

Repository-Metriken

Stars
 (4.293 Sterne)
PR-Merge-Metriken
 (PR-Metriken ausstehend)

Beschreibung

Summary

Reduce Fusion's fixed N+2 cost and support repeated sampling from the same model while preserving Fusion's panel-analysis-synthesis semantics.

Parent hardening tracker: #2336

Blocked by: #1456, #2855, #2856, #2857, #2858, #2861 Related grounding evidence: #2860

Current behavior and gaps

Fusion executes a parallel panel, optional grounding, one structured judge-analysis call, and one final synthesis call. It currently:

  • normalizes/deduplicates analysis model names, so repeating one model does not create true independent self-fusion samples;
  • always pays the analysis call even when include_analysis=false only suppresses trace output;
  • runs judge analysis and final synthesis sequentially using the same judge model;
  • pays the full panel plus judge stages for easy requests;
  • has only limited evidence that panel-consistency grounding helps; hard filtering is known to hurt contested factual QA and soft weighting remains under evaluation in #2860.

Proposed phases

Repeated sampling

  • Add explicit samples_per_model or candidate-call specifications independent of unique model identity.
  • Apply per-sample temperature/seed/reasoning controls and preserve repeated attempts in trace/usage.
  • Report same-model and heterogeneous-panel operation distinctly.

Judge cost reduction

  • Skip the analysis model call when analysis is genuinely disabled.
  • Evaluate a single structured judge response containing bounded analysis plus final answer, compared against the current two-call pipeline.
  • Keep a fallback compatible with current raw-panel synthesis when structured parsing fails.

Adaptive gate

  • Use Router evidence, a cheap lead/probe answer, or verifier confidence to choose direct return, reduced panel, or full Fusion.
  • Reserve every stage from #1456/#2861 before dispatch.
  • Emit the gate decision and counterfactual budget class in Replay.

Research context

ICLR 2026 Fusion-of-N reports benefits from synthesizing complementary samples rather than selecting one. NeurIPS 2025 evidence on overthinking favors parallel samples over extending one chain. These motivate repeated self-sampling, but not unconditional full-panel spend.

Non-goals

  • Do not change grounding defaults in this issue; #2860 decides off|annotate|weight efficacy.
  • Do not turn Fusion into Best-of-N selection.
  • Do not add step-level tree search or arbitrary tool trajectories.

Acceptance criteria

  • Repeated same-model candidates are represented as distinct calls with complete accounting.
  • include_analysis=false no longer pays an unused analysis call, with corrected iterations/usage.
  • A one-call structured judge mode is evaluated against the two-call baseline for parse reliability, quality, tokens, and latency.
  • Adaptive gating has deterministic direct/reduced/full outcomes and cannot exceed hard budgets.
  • Cached-panel tests isolate judge changes from panel sampling noise.
  • Fixed-budget benchmarks report same-model and heterogeneous panels separately.
  • E2E covers repeated samples, skipped analysis, gate outcomes, partial panels, and fallback.

Contributor Guide