vllm-project/semantic-router

feat: extend the routing DSL with WRP governance-as-code constraints

Open

#2.558 aberto em 15 de jul. de 2026

Ver no GitHub
 (0 comments) (0 reactions) (0 assignees)Go (699 forks)github user discovery
area/corearea/model-selectionarea/researchenhancementhelp wantedpriority/P1roadmapsafety

Métricas do repositório

Stars
 (4.293 stars)
Métricas de merge de PR
 (Métricas PR pendentes)

Description

Motivation

Express Workload–Router–Pool (WRP) governance as versioned, reviewable policy while preserving the repository's separation of concerns: the semantic router enforces per-query eligibility/behavior; the serving/LB control plane enforces endpoint capacity and energy decisions.

Audited upstream baseline (2026-07-15)

  • Current configuration/DSL surfaces already support typed signals (including authorization-related signals), decisions, candidate models, and retention actions.
  • ext_proc can enforce request-side routing/tool visibility and emit downstream metadata.
  • Current main does not have a generic WRP constraint language or a pool/cost/energy solver.
  • pkg/authz resolves upstream credentials; it is not the governance policy engine.
  • Tool execution and pool actuation occur outside the semantic router.

Scope

  1. Define a minimal typed policy extension instead of arbitrary expressions:
    • trusted workload/caller/SLO/residency attributes;
    • allowed logical models/tools and hard request constraints;
    • downstream pool capability/residency/SLO/energy requirements as metadata;
    • explicit soft objectives with priority only after hard constraints.
  2. Compile router-owned clauses into existing classification/decision/ext_proc eligibility and actions.
  3. Compile execution/pool-owned clauses into a versioned, integrity-protected downstream contract.
  4. Validate ownership: reject policies that require the router to guarantee an external action it cannot observe/acknowledge.
  5. Add static conflict/unsatisfiable-policy diagnostics and deterministic precedence.
  6. Record policy version, matched rules, hard-constraint result, emitted hints, downstream acknowledgement, and outcome without sensitive payloads.
  7. Provide dry-run, shadow diff, staged rollout, rollback, and migration tooling.

Boundaries

  • Authorization, safety, residency, context compatibility, and explicit denials are hard constraints.
  • Cost/latency/energy preferences cannot weaken a hard constraint.
  • No general-purpose programming language or synchronous solver on the hot path.
  • Tool-call final authorization remains at the execution gateway (#2547).
  • Capacity/energy endpoint choice remains at the LB/control plane (#2513).
  • Caller-controlled request fields cannot assert trusted governance attributes.

Acceptance criteria

  • Schema is closed, typed, versioned, and rejects unknown/ambiguous fields.
  • Every field has an owner, trust source, enforcement point, and acknowledgement semantics.
  • Conflict and infeasibility diagnostics are deterministic and content-free.
  • Dry-run/shadow diff can compare old and new policy before activation.
  • Replay explains matched rules without leaking prompt/tool arguments.
  • E2E tests cover spoofing, denial precedence, unsupported downstream capability, partial deployment, reload, and rollback.
  • Existing configurations remain compatible unless an explicit migration is provided.

Likely change surfaces

src/semantic-router/pkg/config/, src/semantic-router/pkg/classification/, DSL/decision validation, src/semantic-router/pkg/extproc/, src/semantic-router/pkg/routerreplay/, downstream gateway/LB contracts, CLI diagnostics, docs, and tests.

Related: #2357, #2361, #2513, #2546, #2547, #2553, #2557.

Guia do colaborador