vllm-project/semantic-router

Harden the Router Learning runtime contract

Open

#2,239 opened on Jun 18, 2026

View on GitHub
 (0 comments) (0 reactions) (0 assignees)Go (699 forks)github user discovery
area/corearea/docsarea/model-selectionarea/multi-turnenhancementhelp wantedpriority/P2roadmap

Repository metrics

Stars
 (4,293 stars)
PR merge metrics
 (PR metrics pending)

Description

Parent: #2238
Implementation baseline: merged PR #2258

Goal

Harden and document the typed Router Learning runtime contract that exists on current main so future strategies extend one bounded pipeline instead of reviving method-keyed public APIs.

Audited current-main baseline (2026-07-15)

Current code provides:

  • public config under global.router.learning with default adaptation.strategy: routing_sampling, protection, and per-decision adaptations;
  • a fixed pipeline: protection preflight → adaptation strategy → protection switch;
  • a strategy registry and typed routerLearningPolicy/detail serialization;
  • an in-process routerLearningRuntime that owns model experience and accepts typed outcomes through routerruntime.LearningRuntime;
  • replay/header diagnostics at explicit serialization boundaries.

Current code does not contain the previously described routerLearningAdapter interface or a public method-keyed adapter contract. Historical names such as bandit, Elo, session-aware, personalization, RL-driven, GMTRouter, and lookup tables are not the clean public API.

Remaining work

  1. Document package ownership and extension seams for config, strategy registration, runtime state, outcomes, and replay diagnostics.
  2. Make the one-strategy extension checklist explicit: config value/validation, typed strategy implementation, candidate/protection compliance, diagnostics, state bounds, and tests.
  3. Keep loose maps at compatibility/serialization boundaries only; runtime decisions and details remain typed.
  4. Define runtime construction, reload, shutdown, missing-state, and failure semantics.
  5. Prove the hot path uses bounded local state only and does not synchronously read durable replay/storage.
  6. Verify decision bypass, observe/apply modes, candidate sets, hard constraints, and protection ordering cannot be bypassed by a future strategy.
  7. Decide which internal types belong in extproc versus a narrower reusable runtime package; avoid expanding hotspot responsibilities.

Acceptance criteria

  • One documented runtime/strategy contract matches current code and proposal terminology.
  • No documentation or test advertises legacy method-keyed public learning APIs.
  • Future strategies cannot bypass candidate, decision-mode, or protection boundaries.
  • State ownership, lifecycle, reload, missing-state, and fail-open behavior are deterministic.
  • Replay/header serialization is typed before the final map/JSON boundary.
  • Contract tests cover unknown strategy, observe/apply/bypass, protection ordering, reload, and failure paths.
  • Hot-path latency/allocation and state cardinality are bounded and measured.

Likely change surfaces

src/semantic-router/pkg/config/learning_config.go, validator_learning.go, src/semantic-router/pkg/extproc/router_learning_*.go, src/semantic-router/pkg/routerruntime/, Router Replay diagnostics, proposal docs, and tests.

Validation entrypoint

make agent-report ENV=cpu CHANGED_FILES="<space-separated changed files>"

Contributor guide