Summary

Implement the repository-native v0alpha1 Mixture-of-Models lifecycle: build a deterministic composite-model artifact, import/export it by digest, resolve it against an environment, and serve and evaluate the same versioned model identity.

Parent roadmap: #2287

Runtime and lifecycle dependencies: #2331, #2354, #2326, #2358, #1456

Related work: #2333, #2340, #2360, #2366, #2367, #2378, #2393, #2396, #2472

This is an umbrella H2 issue. Schema, verifier, and content-store work can land before all runtime dependencies. Serving, activation, evaluation, replay, and promotion must integrate with existing contracts instead of creating parallel control planes.

Product boundary

A MoM is a virtual composite model, not a router configuration and not agent-harness glue. It presents one ordinary model identity and interface while a lifecycle and execution engine admits and executes finite traces over independently versioned constituents.

The target role for vLLM Semantic Router is:

A lifecycle and execution engine for virtual composite models.

Agents remain clients: they own goals, task decomposition, and application state. The MoM engine owns model-level admissibility, bounded composition, target resolution, resource accounting, and trace attribution.

Audited current baseline

Current main already provides useful primitives:

• canonical v0.3 configuration separates routing semantics from provider access;
• typed model cards, signals, projections, decisions, model references, workflows, plugins, and output contracts;
• provider backend_refs and standard model aliases expose routed and multi-call behavior behind one model API;
• bounded Looper/Flow/Fusion/ReMoM execution;
• Router Replay, usage, system evaluation, and offline recipe-learning hooks; and
• DSL compile/decompile as an authoring frontend.

It does not currently provide:

• a portable MoM manifest or complete canonical typed MoM IR;
• deterministic bundle build/import/export, local CAS, or registry transport;
• behavior variants separated from realization profiles;
• environment binding, capability resolution, or a resolution lock;
• constituent deployment drivers for CUDA, ROCm, CPU, edge, or managed targets;
• bundle/binding/lock identities propagated through serving, replay, and evaluation;
• bundle-aware evaluation attestations or cross-environment conformance; or
• a migration for the legacy “MoM Model Family” terminology used for router-internal auxiliary models.

vllm-sr config import imports OpenClaw configuration into canonical YAML; it is not a MoM artifact importer. Current serve --platform amd|nvidia selects the Semantic Router's own ROCm/CUDA image and router-internal model placement; CPU is the default path rather than a --platform cpu value. It does not provision constituent LLM endpoints.

Required object model

The model format is a typed envelope above leaf formats such as safetensors, ONNX, GGUF, immutable repository snapshots, OCI objects, and opaque provider references. It must distinguish these objects:

Object	Owner	Portable	Meaning
Behavior variant	model author	yes	accuracy-, cost-, speed-, grounding-, or security-first semantics
Request-preference domain	model author	yes	typed per-request refinements allowed by the selected behavior variant
Typed semantic core	model author	yes	interface, graph, policies, constraints, contracts, and bounds
Bundle	publisher	yes	immutable distribution closure around the semantic core
Realization profile	publisher/operator	yes	endpoint-free CUDA, ROCm, CPU, edge, cloud, or hybrid requirements
Environment inventory	environment owner	no	discoverable existing and provisionable targets supplied to bind
Binding	environment owner	no	candidates, endpoints, placement, capacity, pricing, and secret references
Resolution lock	resolver	no	complete resolved candidate inventory and exact evidence
Run record	serving environment	no	one realized trace and its outcomes
Attestation	evaluator/publisher	independently portable	eval result, provenance, SBOM, or signature linked by digest

Names such as vllm-sr/mom-v1-ultra:1.0.0, mom-v1-light:1.0.0, mom-v1-flash:1.0.0, mom-v1-halu:1.0.0, and mom-v1-secu:1.0.0 are distinct logical models when their default objectives, guards, policies, or graphs differ. CUDA and ROCm are realization choices for those models, not model identities.

Backend abstraction

Do not add one overloaded root backend field. A candidate has six independent axes:

1. wire transport (HTTP(S), gRPC, or in-process);
2. API adapter/dialect (for example OpenAI Chat/Responses or Anthropic Messages);
3. model runtime;
4. leaf-artifact format or provider alias;
5. accelerator API/vendor/architecture; and
6. deployment driver.

One trace may combine a ROCm-vLLM local generalist, CUDA specialist, CPU verifier, and managed cloud reasoner. A candidate is admissible only in the intersection of semantic hard constraints, realization-profile requirements, and environment policy. Binding may strengthen a constraint but never weaken it.

Portable and environment boundaries

Portable bundle:

manifest.json
mom.ir.json
source/                       # optional authoring source
assets/ policies/ contracts/
eval/protocols/               # procedures, never results
targets/                      # endpoint-free realization templates
MODEL_CARD.md                 # optional

External records:

fleet.inventory.yaml            # bind input; H2 discovery contract
deployment.binding.yaml             # bind output
resolution.lock.json
run records and private traces
evaluation/build/signature/SBOM attestations

The bundle must not contain secret values, production endpoints, device identifiers, mutable health/capacity, private traces, or online-learning counters. A binding may contain secret references; secret values remain in an external secret store and never enter the bundle or lock.

Support three deterministic export closures:

• thin: immutable external references;
• materialized: redistributable leaf objects are included; and
• targeted: materialized objects plus runtime artifacts for selected realization profiles.

Materialization must fail for a closed provider or a dependency whose license forbids redistribution.

Identity model

Use four acyclic identities:

• semanticDigest: canonical typed IR and its content-addressed semantic assets;
• bundleDigest: canonical manifest and distributed object descriptors;
• bindingDigest: parsed, defaulted, typed, canonical binding IR; and
• lockDigest: canonical completed resolution state.

Never hash raw YAML formatting as environment identity, and no object stores its own digest. Eval results, signatures, SBOMs, and build provenance are external attestations whose subjects include the relevant identities.

Constituent evidence levels must distinguish content-addressed, repository-revision, provider-version, observed-opaque, and floating. Production locking rejects floating; an opaque provider alias must not be presented as bitwise reproducible.

Lifecycle and CLI contract

Build is compilation. Import is verification and registration of an already-built artifact. Export is deterministic materialization.

vllm-sr mom build model.mom.yaml --output model.mombundle
vllm-sr mom build --from-config config.yaml --output model.mombundle
vllm-sr mom import model.mombundle
vllm-sr mom inspect vllm-sr/mom-v1-ultra:1.0.0
vllm-sr mom validate vllm-sr/mom-v1-ultra:1.0.0
vllm-sr mom diff old-ref new-ref

vllm-sr mom bind vllm-sr/mom-v1-ultra:1.0.0 \
  --realization rocm-mi300x --inventory fleet.inventory.yaml \
  --output deployment.binding.yaml
vllm-sr mom resolve vllm-sr/mom-v1-ultra:1.0.0 \
  --binding deployment.binding.yaml --output resolution.lock.json

vllm-sr serve --model vllm-sr/mom-v1-ultra:1.0.0 \
  --binding deployment.binding.yaml --lock resolution.lock.json
vllm-sr mom eval vllm-sr/mom-v1-ultra:1.0.0 \
  --binding deployment.binding.yaml --lock resolution.lock.json \
  --protocol accuracy

vllm-sr mom export vllm-sr/mom-v1-ultra:1.0.0 --format directory
vllm-sr mom push vllm-sr/mom-v1-ultra:1.0.0 oci://registry.example/mom
vllm-sr mom pull oci://registry.example/mom@sha256:...

Production serving requires a lock. Evaluation consumes the same bundle, binding, and lock identities as serving and emits a separate digest-bound attestation.

Implementation strategy

Preserve the current hot path:

DSL / canonical YAML / future frontend
                  ↓
          portable typed MoM IR
                  ↓
      capability-checked lowering
                  ↓
 canonical v0.3 serving realization
                  ↓
       existing router hot path

The DSL remains an optional authoring frontend until it losslessly represents all behavior-affecting runtime fields. Unknown semantics and unsupported operators fail closed; successful builds emit a lowering report.

Separate resolution from provisioning:

TargetResolver
  probe(environment) -> typed capabilities
  resolve(requirement, profile, policy) -> candidates + explanation

DeploymentDriver
  plan(candidate) -> immutable deployment plan
  materialize(plan) -> external references
  observe(plan) -> endpoint and revision evidence

The first driver should support existing endpoints, matching current behavior. Add vLLM CUDA, vLLM ROCm, CPU, Kubernetes, edge/NPU, and managed-provider drivers incrementally without changing the model format.

H2 milestones

M1 — Format alpha

• ADR for Bundle, IR, Behavior Variant, Request Preference, Realization Profile, Environment Inventory, Binding, Lock, Run Record, and Attestation.
• Language-neutral schemas, media types, canonicalization, deterministic packaging, negative fixtures, and thin/materialized/targeted closures.
• Legacy auxiliary-model terminology migration to “Semantic Router system models” / RouterModelRegistry with compatibility aliases.

M2 — Build, import, and distribution

• Semantic-closure compiler from canonical v0.3 and DSL; unknown semantics fail closed.
• Local CAS and immutable name/version/tag-to-digest registry.
• Build/import/export/inspect/diff plus OCI pull/push and referrer support.

M3 — Bind, lock, and serve

• Engine and target capability descriptors with required/preferred negotiation and diagnostics.
• Canonical environment inventory, binding, and lock with an existing-endpoint driver.
• Locked IR-to-canonical-v0.3 lowering and versioned model IDs through the standard API.
• vLLM CUDA, vLLM ROCm, and CPU deployment drivers, then edge/NPU and managed integrations.

M4 — Evaluate and optimize

• All four identities in response metadata, trace, replay, and learning records.
• Bundle/binding/lock-aware system evaluation and digest-bound result attestations.
• Candidate promotion, rollback, and cross-environment conformance.

Completion criteria

• Deterministic build and independent verification produce stable semantic and bundle identities.
• Source/binding formatting changes do not alter semantic/binding identity.
• Import/export round-trips supported semantics or reports a typed incompatibility; behavior is never silently dropped.
• Portable bundles contain no credentials, secret values, live health, counters, or private traces.
• Every reachable reference resolves to an admissible candidate or explicit fallback, abstention, or rejection.
• Resolution fails closed for missing capabilities, hard-constraint violations, invalid bounds, or unknown required evidence.
• Locks contain the complete candidate inventory and exact model/tokenizer/runtime/image/accelerator evidence.
• A standard model request selects a versioned MoM coordinate and produces one contracted response.
• Serving, replay, and evaluation carry semantic, bundle, binding, and lock identities.
• Positive and negative conformance fixtures run in CI.
• User docs cover build, import, export, bind, resolve, serve, eval, rebind, promote, and rollback.

The H2 exit criterion is concrete:

The same immutable MoM bundle can be imported, served, and evaluated once under a CUDA binding and once under a ROCm binding. The runs retain the same semantic and bundle identities, receive different binding and lock identities, and make every response, trace, and evaluation attestation attributable to all four.

Non-goals

• Do not turn the semantic router into an unbounded agent or arbitrary workflow engine.
• Do not execute undeclared or untrusted plugin code because it appears in a bundle.
• Do not package credentials, mutable endpoint state, or private traces.
• Do not require all constituent weights to be copied into the bundle.
• Do not claim that rebinding yields identical outputs; portability preserves declared logical/control semantics and makes substitutions observable.
• Do not replace downstream load balancing, endpoint admission, or FleetSim.
• Do not enable automatic online promotion without approval, canary, and rollback contracts.

Likely implementation surfaces

schema/mom/v0alpha1/
src/semantic-router/pkg/momartifact/
src/semantic-router/pkg/config/
src/semantic-router/pkg/dsl/
src/semantic-router/pkg/routerreplay/
src/vllm-sr/cli/mom/
deploy/recipes/
e2e/
website/docs/

Keep new modules narrow; do not grow config, CLI, or extproc hotspot files into bundle subsystems. Every implementation slice must use make agent-report for its actual changed-file surface, then run the smallest applicable validation, CI, feature, and affected-E2E gates.