vllm-project/semantic-router

feat: extend the routing DSL with WRP governance-as-code constraints

Open

#2558 aperta il 15 lug 2026

Vedi su GitHub
 (0 commenti) (0 reazioni) (0 assegnatari)Go (699 fork)github user discovery
area/corearea/model-selectionarea/researchenhancementhelp wantedpriority/P1roadmapsafety

Metriche repository

Star
 (4293 star)
Metriche merge PR
 (Metriche PR in attesa)

Descrizione

Motivation

Express Workload–Router–Pool (WRP) governance as versioned, reviewable policy while preserving the repository's separation of concerns: the semantic router enforces per-query eligibility/behavior; the serving/LB control plane enforces endpoint capacity and energy decisions.

Audited upstream baseline (2026-07-15)

  • Current configuration/DSL surfaces already support typed signals (including authorization-related signals), decisions, candidate models, and retention actions.
  • ext_proc can enforce request-side routing/tool visibility and emit downstream metadata.
  • Current main does not have a generic WRP constraint language or a pool/cost/energy solver.
  • pkg/authz resolves upstream credentials; it is not the governance policy engine.
  • Tool execution and pool actuation occur outside the semantic router.

Scope

  1. Define a minimal typed policy extension instead of arbitrary expressions:
    • trusted workload/caller/SLO/residency attributes;
    • allowed logical models/tools and hard request constraints;
    • downstream pool capability/residency/SLO/energy requirements as metadata;
    • explicit soft objectives with priority only after hard constraints.
  2. Compile router-owned clauses into existing classification/decision/ext_proc eligibility and actions.
  3. Compile execution/pool-owned clauses into a versioned, integrity-protected downstream contract.
  4. Validate ownership: reject policies that require the router to guarantee an external action it cannot observe/acknowledge.
  5. Add static conflict/unsatisfiable-policy diagnostics and deterministic precedence.
  6. Record policy version, matched rules, hard-constraint result, emitted hints, downstream acknowledgement, and outcome without sensitive payloads.
  7. Provide dry-run, shadow diff, staged rollout, rollback, and migration tooling.

Boundaries

  • Authorization, safety, residency, context compatibility, and explicit denials are hard constraints.
  • Cost/latency/energy preferences cannot weaken a hard constraint.
  • No general-purpose programming language or synchronous solver on the hot path.
  • Tool-call final authorization remains at the execution gateway (#2547).
  • Capacity/energy endpoint choice remains at the LB/control plane (#2513).
  • Caller-controlled request fields cannot assert trusted governance attributes.

Acceptance criteria

  • Schema is closed, typed, versioned, and rejects unknown/ambiguous fields.
  • Every field has an owner, trust source, enforcement point, and acknowledgement semantics.
  • Conflict and infeasibility diagnostics are deterministic and content-free.
  • Dry-run/shadow diff can compare old and new policy before activation.
  • Replay explains matched rules without leaking prompt/tool arguments.
  • E2E tests cover spoofing, denial precedence, unsupported downstream capability, partial deployment, reload, and rollback.
  • Existing configurations remain compatible unless an explicit migration is provided.

Likely change surfaces

src/semantic-router/pkg/config/, src/semantic-router/pkg/classification/, DSL/decision validation, src/semantic-router/pkg/extproc/, src/semantic-router/pkg/routerreplay/, downstream gateway/LB contracts, CLI diagnostics, docs, and tests.

Related: #2357, #2361, #2513, #2546, #2547, #2553, #2557.

Guida contributor