vllm-project/semantic-router

feature: add per-step controls and accounting to static Router Flow workflows

Open

#1,815 opened on 2026年4月23日

GitHub で見る
 (4 comments) (0 reactions) (0 assignees)Go (699 forks)github user discovery
area/model-selectionarea/researchenhancementhelp wantedpriority/P2roadmap

Repository metrics

Stars
 (4,293 stars)
PR merge metrics
 (PR metrics pending)

説明

Audited upstream baseline (2026-07-15)

The original gap is partly implemented on current main. A decision can use algorithm.type: workflows with mode: static; ordered roles declare model lists, prompts, and access lists, followed by a configured final model. The runtime also has workflow-wide max_steps, round_timeout_seconds, on_error, concurrency, and intermediate-response controls.

The remaining gap is a narrow fixed-pipeline contract with explicit controls and receipts for each ordered step. This issue must extend the existing static Router Flow implementation, not introduce a second pipeline engine.

Target use case

Example: summarize with model A, extract structured fields with model B from the summary, then return a final response. Operators should be able to reason about the model, input dependency, timeout, failure behavior, output contract, and resource use of each step.

Scope

  • Add stable per-step identity and explicit single-model pinning where a fixed pipeline requires it.
  • Define how a step consumes reviewed outputs from earlier steps.
  • Add per-step timeout, failure/skip policy, token limit, and optional output contract.
  • Emit per-step latency, token, cost, model, status, and failure receipts to diagnostics/Replay.
  • Preserve workflow-wide safety, authorization, cancellation, and maximum-amplification limits.
  • Provide migration/examples using the existing workflows config surface.

Boundaries

  • Do not create an arbitrary user-programmable DAG or external workflow engine.
  • Do not duplicate static Workflows under a new top-level API.
  • Do not allow a step to bypass the decision candidate set, plugins, or provider policy.
  • Existing single-step routing and existing static/dynamic workflow configs remain compatible.

Acceptance criteria

  • A two-step A-then-B pipeline is expressible through the existing Workflows contract.
  • Each step has deterministic model/input/error semantics.
  • Per-step and aggregate usage are both observable.
  • Cancellation and timeout stop downstream work safely.
  • Tests cover success, timeout, skipped/failed step, output-contract failure, and streaming behavior.
  • Docs clearly distinguish fixed static pipelines from dynamic workflows and hierarchical recipe routing (#2343).

Relevant surfaces

src/semantic-router/pkg/config/workflows_config.go, src/semantic-router/pkg/looper/workflows*.go, Replay/diagnostics, docs/examples, and E2E tests.

コントリビューターガイド