vllm-project/semantic-router

feature: add bounded verification and repair to Router Flow

Ouverte

#2 867 ouverte le 11 août 2026

 (0 commentaire) (0 réaction) (0 personne assignée)Go (699 forks)github user discovery
area/agentarea/corearea/momarea/researcharea/tool-managementenhancementhelp wantedpriority/P2roadmap

Métriques du dépôt

Stars
 (4 293 étoiles)
Métriques de merge PR
 (Métriques PR en attente)

Description

Summary

Add explicit verification, bounded repair, and context compaction to Router Flow while preserving workflow state ownership, per-step controls, and hard request budgets.

Parent hardening tracker: #2336

Blocked by: #1456, #1815, #2471, #2855, #2857, #2859, #2861

Current behavior and gaps

Workflows supports static roles or a dynamic planner, sequential steps with parallel models inside a step, final synthesis, and stateful tool pause/resume. However:

  • verifier, reviewer, and critic are prompt roles rather than typed checks with scores/results;
  • failed or weak outputs do not enter a generic verify-repair loop with an explicit budget;
  • accessible prior step outputs are repeatedly embedded in later prompts and can grow context quickly;
  • steps execute sequentially even when dependencies could eventually permit independent scheduling;
  • production tool-resume semantics depend on the state lifecycle/durability work in #2471;
  • stable per-step timeout/token/output receipts belong with #1815.

Proposed phases

Typed verification

  • Add workflow step/final verification configuration backed by #2857.
  • Prefer deterministic signals: parser/schema validation, executable tests, sandbox results, tool-result contracts, or trusted-source checks.
  • Use LLM judges only as calibrated outcome verifiers, not as unbounded free-form critics.

Bounded repair

  • On a failed verification result, generate a targeted repair request containing the original contract, bounded failure evidence, and permitted prior outputs.
  • Limit attempts, tokens, cost, calls, and wall time via #1456/#2861.
  • Define fail, skip, fallback, and abstain outcomes when repair is exhausted.
  • Persist pause/resume state and exactly-once tool turns through #2471.

Context compaction

  • Add per-step output summaries or bounded reference views instead of replaying every full prior response.
  • Preserve output-contract fields and trusted tool/test evidence losslessly.
  • Record before/after token estimates and omitted references.

Dependency-aware scheduling (later slice)

  • Derive a DAG from explicit access/dependency declarations.
  • Run only independent ready steps concurrently.
  • Keep this behind static-plan validation and do not merge it into the first verification/repair PR.

Research context

Tool-verified execution is strongest where the environment supplies objective feedback, such as code tests, SQL execution, schemas, and agent tools. ICML 2025 JETTS cautions that natural-language critique alone is ineffective for guiding refinement; repair should therefore be anchored in typed or executable evidence whenever possible.

Non-goals

  • Do not create a general external workflow engine.
  • Do not bypass decision candidate sets, authorization, plugins, or tool policy.
  • Do not duplicate #2471 state ownership or #1815 per-step accounting.
  • Do not permit unbounded autonomous retries.

Acceptance criteria

  • Static and dynamic plans can declare typed verifier and repair policies with validation.
  • Deterministic tool/test/schema verification is supported end to end.
  • Repair attempts are bounded and every verify/repair/tool turn is traced and budgeted.
  • Exhaustion has deterministic fail/skip/fallback/abstain semantics.
  • Context compaction reduces repeated prompt tokens without losing required contracts/evidence.
  • Two-request tool pause/resume remains exactly once and durable under #2471 semantics.
  • Per-step and aggregate receipts align with #1815 and #2855.
  • Benchmark slices compare unverified, verified, and verify-repair Flow on executable agentic tasks.
  • DAG scheduling, if implemented, lands as a later independently reviewable slice.

Guide contributeur