vllm-project/semantic-router

feature: implement the portable, content-addressed MoM Bundle lifecycle

Open

#2,562 opened on Jul 15, 2026

View on GitHub
 (0 comments) (0 reactions) (0 assignees)Go (699 forks)github user discovery
area/corearea/momarea/observabilityarea/test-and-releasearea/user-experienceenhancementhelp wantedpriority/P1roadmap

Repository metrics

Stars
 (4,293 stars)
PR merge metrics
 (PR metrics pending)

Description

Summary

Implement the first repository-native v0alpha1 MoM Bundle lifecycle: compile canonical YAML or DSL into a deterministic portable artifact, verify its content identities and compatibility, bind logical references to an environment, resolve an immutable lock, and attribute every realized request trace to the exact logical model, distributed bundle, environment binding, and resolution.

Parent roadmap: #2287

Runtime-serving dependencies: #2331, #2354, #2326, #2358, #1456

Related: #2333, #2340, #2360, #2366, #2367, #2378, #2393, #2396, #2472

This is a parent implementation issue. Schema, verifier, and CLI work can land before all runtime dependencies; serving, activation, evaluation, and promotion must integrate with the existing contracts instead of creating parallel ones.

Audited baseline

Current main already provides useful building blocks:

  • canonical v0.3 YAML with routing semantics separated from provider access;
  • DSL compile/decompile for routing-owned semantics;
  • typed modelCards, modelRefs, provider backend_refs, capabilities, pricing, and external model identifiers;
  • output contracts, maintained recipes, and bounded Looper-family composition; and
  • Router Replay records for decisions, signals, projections, selected models, usage, cost, outcomes, and tool traces.

It does not currently provide:

  • a portable MoM manifest or canonical typed MoM IR artifact;
  • deterministic bundle build, inspect, verify, import, or export commands;
  • logical, bundle, binding, and lock content identities;
  • a standalone environment-owned binding and completed resolution lock;
  • engine/operator compatibility negotiation;
  • trace attribution to all four identities; or
  • a secret-safe bundle boundary, signatures/attestations, and cross-engine conformance fixtures.

vllm-sr config import currently imports OpenClaw models into canonical YAML; it is not a MoM artifact importer. Provider configuration can also carry secret values, so it must not be copied into a portable bundle.

User outcome

A user can build and publish a model-like coordinate such as vllm-sr/mom-v1-ultra@1.0.0, then bind it independently on a laptop, at an edge site, in a private data center, or in a cloud environment. To callers it remains an ordinary model selected through the standard API model field; the runtime records exactly which bundle, binding, lock, constituents, operators, and contracts produced the response.

Illustrative CLI contract:

vllm-sr mom bundle build \
  --config deploy/recipes/accuracy-first.yaml \
  --model-id vllm-sr/mom-v1-ultra \
  --version 1.0.0 \
  --output mom-v1-ultra.mom

vllm-sr mom bundle inspect mom-v1-ultra.mom
vllm-sr mom bundle verify mom-v1-ultra.mom
vllm-sr mom bind mom-v1-ultra.mom --binding deployment.binding.yaml
vllm-sr mom resolve mom-v1-ultra.mom \
  --binding deployment.binding.yaml \
  --output resolution.lock.json

Exact command names may change during implementation, but build, inspect, verify, bind, and resolve must remain distinct operations.

Artifact boundary

The portable bundle contains only immutable, distribution-safe records:

manifest.json             model coordinate, entrypoints, object index, compatibility
mom.ir.json               canonical executable MoM specification
source/                   optional authoring YAML or DSL
assets/                   prompts, controller state, schemas, semantic assets
contracts/                typed request/response contracts
eval/                     optional evaluation summaries
provenance/               build and source provenance

Environment-owned records remain outside the portable bundle:

deployment.binding.yaml   admissible candidates and secret references
resolution.lock.json      selected revisions, protocols, evidence, engine revision
realized traces           request-time events, checks, cost, latency, outcomes

The identity model must avoid circular self-hashes:

  • logicalDigest: canonical typed IR plus path-sorted semantic assets;
  • bundleDigest: manifest bytes plus every manifest-referenced distributed object;
  • bindingDigest: exact environment binding bytes; and
  • lockDigest: exact completed resolution-lock bytes.

The manifest stores logicalDigest; the completed lock stores the bundle and binding digests; no object stores its own digest.

Scope

1. Versioned schemas and deterministic compilation

  • Define v0alpha1 schemas for the manifest, canonical MoM IR, binding, resolution lock, and trace identity fields.
  • Compile supported canonical YAML/DSL semantics into one IR authority rather than inventing a second runtime configuration model.
  • Represent entrypoints, logical model requirements, signals, projections, preference profiles, hard constraints, bounded operators/graphs, contracts, assets, and finite resource bounds.
  • Materialize defaults, reject unknown fields, normalize relative POSIX paths, and produce deterministic canonical bytes.
  • Preserve semantic round-trip behavior where the current YAML/DSL surface can express the same construct; reject unsupported constructs explicitly.

2. Portable bundle build and verification

  • Add build, inspect, verify, import, export, and semantic-diff support to the CLI.
  • Compute and verify logical and bundle digests.
  • Reject absolute paths, traversal, backslashes, repeated separators, symlinks, digest mismatches, duplicate paths, missing objects, incompatible engine/operator requirements, and undeclared assets.
  • Keep credentials, endpoint health, online counters, mutable learning state, private traces, and raw provider secrets out of the bundle.
  • Permit large weights to remain external only through explicit content-addressed references and declared license/provenance records.

3. Environment binding and resolution lock

  • Map every reachable logical model/operator reference to one or more environment candidates, or to an explicit fallback, abstention, or rejection path.
  • Declare protocol, placement and device class, capability evidence, capacity envelope, pricing assumptions, data policy, and secretRef without embedding secret values.
  • Fail closed when a required capability, constraint, or revision fact is unknown.
  • Record engine and constituent revisions in the lock.
  • Distinguish immutable available revision evidence from an opaque provider alias observed at a specific time; never imply bitwise replay for opaque services.

4. Bounded action-graph and runtime identity integration

  • Compile the supported typed MoM graph onto existing selection, Fusion, Workflows, ReMoM, contract, and plugin primitives instead of building a separate workflow engine.
  • Validate reachability, terminal states, references, contracts, and finite limits for events, fan-out, model calls, rounds, tokens, and wall time.
  • Resolve a versioned model coordinate and default profile through #2331/#2354, then serve the locked bundle through the ordinary model API.
  • Add logical, bundle, binding, and lock identities to request context, Router Replay, diagnostics, evaluation receipts, and distributed traces without exposing internal identifiers as public response headers by default.
  • Preserve reconstructable event/fork/join relationships for multi-call traces.

5. Lifecycle integration

  • Integrate activation and rollback with #2326 rather than adding an unrelated mutable configuration path.
  • Allow evaluation and offline optimization to emit a complete candidate bundle/version; promotion creates a new immutable version and never mutates the live bundle in place.
  • Keep real-time endpoint scheduling, queueing, spillover, and replica admission in downstream serving/LB control planes. A binding can declare candidates and capacity evidence, but does not turn the semantic router into a fleet scheduler.

6. Conformance, trust, and documentation

  • Add one complete credential-free fixture containing source, IR, semantic assets, a contract, evaluation/provenance records, binding, available and opaque revision evidence, and a valid lock.
  • Add negative fixtures for traversal, secret leakage, unknown fields, digest drift, missing references, incompatible operators, unsatisfied constraints, unbounded graphs, and incomplete locks.
  • Define follow-up slices for manifest signatures, attestations, plugin interface/digest/license policy, and cross-engine conformance before closing this parent issue if they do not land in the initial implementation.
  • Document the portable/environment boundary, identities, compatibility, CLI, and migration from canonical v0.3 configuration.

Non-goals

  • Do not turn the semantic router into an unbounded agent or arbitrary workflow engine.
  • Do not execute undeclared or untrusted plugin code merely because it appears in a bundle.
  • Do not package credentials, mutable endpoint health, private request traces, or online learning counters.
  • Do not require all constituent weights to be copied into the bundle.
  • Do not claim that rebinding produces identical outputs; portability preserves declared logical/control semantics and makes substitutions observable.
  • Do not replace downstream load balancing, endpoint admission, or FleetSim.
  • Do not enable automatic online promotion without the approval and rollback work in #2366/#2367.

Likely implementation surfaces

src/semantic-router/pkg/config/
src/semantic-router/pkg/dsl/
src/semantic-router/pkg/routerreplay/
src/semantic-router/pkg/extproc/
src/vllm-sr/cli/
deploy/recipes/
e2e/
website/docs/

Prefer a focused new package such as src/semantic-router/pkg/momartifact/ for schemas, canonicalization, digest computation, archive/path safety, binding validation, and lock construction. Do not grow config or CLI hotspot files into bundle subsystems.

Completion criteria

  • The same normalized logical semantics and semantic assets always produce the same logicalDigest.
  • Source formatting changes preserve logicalDigest; evaluation- or provenance-only changes alter bundleDigest without changing logical identity.
  • A deterministic bundle build produces stable identities and can be independently verified.
  • Manifest, IR, binding, lock, and trace schemas reject unknown or unsafe input.
  • Import/export round-trips supported semantics or reports a typed incompatibility; it never silently drops behavior.
  • Portable bundles contain no credentials, secret values, mutable health, online counters, or private traces.
  • Every reachable reference resolves to an admissible candidate or an explicit fallback, abstention, or rejection edge.
  • Resolution fails closed for missing capabilities, violated data policy, incompatible operators, invalid bounds, or unknown required evidence.
  • Locks record engine and constituent revisions and distinguish available from opaque revision evidence.
  • An ordinary model request can select a pinned MoM coordinate and produce one contracted response.
  • Router Replay and trace output carry logical, bundle, binding, and lock identities plus resolved calls and terminal status.
  • At least one positive fixture and the required negative fixtures run in CI.
  • User docs explain build, verify, bind, resolve, serve, inspect, rebind, promotion, and rollback.
  • Any intentionally deferred action-graph, signature, attestation, or cross-engine work is split into linked H2 child issues before this parent closes.

Validation

Start every implementation slice by resolving its actual changed-file surface:

make agent-report ENV=cpu CHANGED_FILES="src/semantic-router/pkg/momartifact src/semantic-router/pkg/config src/semantic-router/pkg/dsl src/semantic-router/pkg/routerreplay src/semantic-router/pkg/extproc src/vllm-sr/cli deploy/recipes e2e website/docs"
make agent-validate
make agent-lint CHANGED_FILES="src/semantic-router/pkg/momartifact src/semantic-router/pkg/config src/semantic-router/pkg/dsl src/semantic-router/pkg/routerreplay src/semantic-router/pkg/extproc src/vllm-sr/cli deploy/recipes e2e website/docs"
make agent-ci-gate CHANGED_FILES="src/semantic-router/pkg/momartifact src/semantic-router/pkg/config src/semantic-router/pkg/dsl src/semantic-router/pkg/routerreplay src/semantic-router/pkg/extproc src/vllm-sr/cli deploy/recipes e2e website/docs"
make vllm-sr-test
make test-semantic-router
make vllm-sr-test-integration
make agent-feature-gate ENV=cpu CHANGED_FILES="src/semantic-router/pkg/momartifact src/semantic-router/pkg/config src/semantic-router/pkg/dsl src/semantic-router/pkg/routerreplay src/semantic-router/pkg/extproc src/vllm-sr/cli deploy/recipes e2e website/docs"
make agent-e2e-affected CHANGED_FILES="src/semantic-router/pkg/momartifact src/semantic-router/pkg/config src/semantic-router/pkg/dsl src/semantic-router/pkg/routerreplay src/semantic-router/pkg/extproc src/vllm-sr/cli deploy/recipes e2e website/docs"

Run the local-image smoke path for behavior-visible activation/serve changes. Use ENV=amd in addition to CPU validation when a slice changes AMD/device binding behavior.

Contributor guide