vllm-project/semantic-router

feature: add bounded verification and repair to Router Flow

Offen

#2.867 geöffnet am 11.08.2026

 (0 Kommentare) (0 Reaktionen) (0 zugewiesene Personen)Go (699 Forks)github user discovery
area/agentarea/corearea/momarea/researcharea/tool-managementenhancementhelp wantedpriority/P2roadmap

Repository-Metriken

Stars
 (4.293 Sterne)
PR-Merge-Metriken
 (PR-Metriken ausstehend)

Beschreibung

Summary

Add explicit verification, bounded repair, and context compaction to Router Flow while preserving workflow state ownership, per-step controls, and hard request budgets.

Parent hardening tracker: #2336

Blocked by: #1456, #1815, #2471, #2855, #2857, #2859, #2861

Current behavior and gaps

Workflows supports static roles or a dynamic planner, sequential steps with parallel models inside a step, final synthesis, and stateful tool pause/resume. However:

  • verifier, reviewer, and critic are prompt roles rather than typed checks with scores/results;
  • failed or weak outputs do not enter a generic verify-repair loop with an explicit budget;
  • accessible prior step outputs are repeatedly embedded in later prompts and can grow context quickly;
  • steps execute sequentially even when dependencies could eventually permit independent scheduling;
  • production tool-resume semantics depend on the state lifecycle/durability work in #2471;
  • stable per-step timeout/token/output receipts belong with #1815.

Proposed phases

Typed verification

  • Add workflow step/final verification configuration backed by #2857.
  • Prefer deterministic signals: parser/schema validation, executable tests, sandbox results, tool-result contracts, or trusted-source checks.
  • Use LLM judges only as calibrated outcome verifiers, not as unbounded free-form critics.

Bounded repair

  • On a failed verification result, generate a targeted repair request containing the original contract, bounded failure evidence, and permitted prior outputs.
  • Limit attempts, tokens, cost, calls, and wall time via #1456/#2861.
  • Define fail, skip, fallback, and abstain outcomes when repair is exhausted.
  • Persist pause/resume state and exactly-once tool turns through #2471.

Context compaction

  • Add per-step output summaries or bounded reference views instead of replaying every full prior response.
  • Preserve output-contract fields and trusted tool/test evidence losslessly.
  • Record before/after token estimates and omitted references.

Dependency-aware scheduling (later slice)

  • Derive a DAG from explicit access/dependency declarations.
  • Run only independent ready steps concurrently.
  • Keep this behind static-plan validation and do not merge it into the first verification/repair PR.

Research context

Tool-verified execution is strongest where the environment supplies objective feedback, such as code tests, SQL execution, schemas, and agent tools. ICML 2025 JETTS cautions that natural-language critique alone is ineffective for guiding refinement; repair should therefore be anchored in typed or executable evidence whenever possible.

Non-goals

  • Do not create a general external workflow engine.
  • Do not bypass decision candidate sets, authorization, plugins, or tool policy.
  • Do not duplicate #2471 state ownership or #1815 per-step accounting.
  • Do not permit unbounded autonomous retries.

Acceptance criteria

  • Static and dynamic plans can declare typed verifier and repair policies with validation.
  • Deterministic tool/test/schema verification is supported end to end.
  • Repair attempts are bounded and every verify/repair/tool turn is traced and budgeted.
  • Exhaustion has deterministic fail/skip/fallback/abstain semantics.
  • Context compaction reduces repeated prompt tokens without losing required contracts/evidence.
  • Two-request tool pause/resume remains exactly once and durable under #2471 semantics.
  • Per-step and aggregate receipts align with #1815 and #2855.
  • Benchmark slices compare unverified, verified, and verify-repair Flow on executable agentic tasks.
  • DAG scheduling, if implemented, lands as a later independently reviewable slice.

Contributor Guide