feature: optional two-stage cross-encoder rerank as a routing signal
#2,251 opened on Jun 18, 2026
Repository metrics
- Stars
- (4,293 stars)
- PR merge metrics
- (PR metrics pending)
Description
Audited upstream baseline (2026-07-15)
Current main does not yet contain a general cross-encoder runtime or rerank endpoint. PR #2236 is open and unmerged. The existing router does have embedding/prototype scoring and bounded candidate sets that can provide the cheap first stage.
This issue begins after the typed model/runtime contract in #2250 is available. It must evaluate whether a cross-encoder improves route decisions; quality or calibration gains are hypotheses, not assumed production facts.
Scope
- Define an optional registered signal that reranks a bounded top-K candidate set.
- Keep the first-stage source explicit (for example a reviewed embedding shortlist).
- Emit typed score, margin, provenance, model version, latency, and fallback reason.
- Define timeout/invalid-output behavior that degrades to the reviewed first-stage result.
- Add decision-level offline/shadow evaluation for route correctness, latency, and cost.
- Keep the feature disabled by default until evaluation supports a preset.
Boundaries
- Do not conflate document/KB reranking with model-route reranking; define the candidate and label semantics.
- Do not call an unbounded set of candidates.
- Do not block unrelated routing when the optional reranker is unavailable.
- Do not bypass candidate authorization, safety policy, or the decision contract.
Acceptance criteria
- The signal is optional and registered/observable through #2330.
- Top-K and timeout limits are validated.
- Missing, stale, or failed reranking has a typed fallback receipt.
- Decision evaluation compares first-stage-only and two-stage routing on representative data.
- Tests cover ranking, ties, timeout, invalid output, disabled state, and model unavailability.
Relevant surfaces
src/semantic-router/pkg/classification, typed signal/config registration, src/semantic-router/pkg/decision, model runtime (#2250), diagnostics, and #2247.