vllm-project/semantic-router
Vedi su GitHubfeat: extend the routing DSL with WRP governance-as-code constraints
Open
#2558 aperta il 15 lug 2026
area/corearea/model-selectionarea/researchenhancementhelp wantedpriority/P1roadmapsafety
Metriche repository
- Star
- (4293 star)
- Metriche merge PR
- (Metriche PR in attesa)
Descrizione
Motivation
Express Workload–Router–Pool (WRP) governance as versioned, reviewable policy while preserving the repository's separation of concerns: the semantic router enforces per-query eligibility/behavior; the serving/LB control plane enforces endpoint capacity and energy decisions.
Audited upstream baseline (2026-07-15)
- Current configuration/DSL surfaces already support typed signals (including authorization-related signals), decisions, candidate models, and retention actions.
- ext_proc can enforce request-side routing/tool visibility and emit downstream metadata.
- Current
maindoes not have a generic WRP constraint language or a pool/cost/energy solver. pkg/authzresolves upstream credentials; it is not the governance policy engine.- Tool execution and pool actuation occur outside the semantic router.
Scope
- Define a minimal typed policy extension instead of arbitrary expressions:
- trusted workload/caller/SLO/residency attributes;
- allowed logical models/tools and hard request constraints;
- downstream pool capability/residency/SLO/energy requirements as metadata;
- explicit soft objectives with priority only after hard constraints.
- Compile router-owned clauses into existing classification/decision/ext_proc eligibility and actions.
- Compile execution/pool-owned clauses into a versioned, integrity-protected downstream contract.
- Validate ownership: reject policies that require the router to guarantee an external action it cannot observe/acknowledge.
- Add static conflict/unsatisfiable-policy diagnostics and deterministic precedence.
- Record policy version, matched rules, hard-constraint result, emitted hints, downstream acknowledgement, and outcome without sensitive payloads.
- Provide dry-run, shadow diff, staged rollout, rollback, and migration tooling.
Boundaries
- Authorization, safety, residency, context compatibility, and explicit denials are hard constraints.
- Cost/latency/energy preferences cannot weaken a hard constraint.
- No general-purpose programming language or synchronous solver on the hot path.
- Tool-call final authorization remains at the execution gateway (#2547).
- Capacity/energy endpoint choice remains at the LB/control plane (#2513).
- Caller-controlled request fields cannot assert trusted governance attributes.
Acceptance criteria
- Schema is closed, typed, versioned, and rejects unknown/ambiguous fields.
- Every field has an owner, trust source, enforcement point, and acknowledgement semantics.
- Conflict and infeasibility diagnostics are deterministic and content-free.
- Dry-run/shadow diff can compare old and new policy before activation.
- Replay explains matched rules without leaking prompt/tool arguments.
- E2E tests cover spoofing, denial precedence, unsupported downstream capability, partial deployment, reload, and rollback.
- Existing configurations remain compatible unless an explicit migration is provided.
Likely change surfaces
src/semantic-router/pkg/config/, src/semantic-router/pkg/classification/, DSL/decision validation, src/semantic-router/pkg/extproc/, src/semantic-router/pkg/routerreplay/, downstream gateway/LB contracts, CLI diagnostics, docs, and tests.
Related: #2357, #2361, #2513, #2546, #2547, #2553, #2557.