Hacktoberfest 2026：メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

Python: [Feature]: A deterministic pre-execution verification middleware for agent actions — AgentDojo v2.2 re-run: ASR=0 / FP=0 (open artifacts, full fix-cycle trajectory inside)

オープン
  • 何か問題がありますか？
#9,132 コメント 0 件 リアクション 0 件 担当者 1 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

@eavanvalkenburg がすでに取り組んでいます。

2026年10月7日 から。

評価

この issue はまだ評価されていません。

説明

agents middleware python
Description

Discussions like #8862 (budget enforcement in AgentLoopMiddleware) and #7853 (sandbox abstraction for tool execution) point at the same gap: agent frameworks today observe actions, but the decision to block a dangerous action before it executes is usually left ad hoc. We built that missing piece and would like to offer it to this community as a candidate middleware contract.

What it is
agent-action-verifier (https://github.com/Lsy1533133/agent-action-verifier) is a deterministic pre-execution enforcement layer: every pending action (tool call, network egress, file write) is checked against the agent's declared plan before execution. The judge is fixed-constant math — no LLM call in the enforcement path, no learned parameters — so it adds microsecond-scale cost per action (P99 = 0.13 ms in the synthetic stress suite; production latency not yet measured) and is deterministic and auditable by construction.

Closed-loop results (each reported number is backed by a checksummed artifact in the repo)

  • AgentDojo official benchmark, real LLM in the loop (97 tasks × 2 rounds, official injection suite, official security() judgment): ASR = 0, FP = 0 in the v2.2 full re-run (FP-rate 95% upper bound ≈3.8% at n=97 benign rounds)
  • Published fix-cycle trajectory FP 16 → 1 → 0 across v1.2 → v2.1 → v2.2 — same-source iteration, failures included, not independent stability trials
  • Cross-model spot check: GLM subset (24 benign + 24 attack): FP = 0, ASR = 0
  • Synthetic stress layer (100,000 seeded scenarios): FN = 0, FP = 0; interception Wilson-95 lower bound 99.99%+ on that synthetic distribution
  • White-box adaptive attacks: 49 cases, 16 adaptive vector families — 43 hard-blocked, 0 bypass; 6 boundary cases documented and not counted as bypasses under the stated threat model

Integration shape (matches your middleware seam)
Wrap the action-dispatch point; the verifier receives (declared plan, pending action) and returns PASS / VETO + rule id. verifier_interface.pyi in the repo specifies the contract; typical adapter is ~10 lines around an AgentLoopMiddleware. Rule families: out-of-plan action, egress breach, scope escalation, tool-consent violation, dangerous value class, arithmetic guard, uninitialised state.

Honest boundaries
Synthetic scenarios are abstracted from publicly disclosed incident categories, not production traffic. Same-source fix cycles ≠ independent stability trials. GLM subset vs full run differ in model and sample size and are not directly comparable. This layer complements monitoring/alignment — it does not replace them.

Verify it yourself
python verify_artifacts.py in the repo recomputes every SHA-256 chain and the Wilson-95 bounds from raw counts — no trust required, stdlib only.

We'd genuinely value feedback on the metrics methodology, and if a deterministic enforcement seam fits the framework's roadmap, if maintainers see a fit, we can align the contract with the framework's middleware design.

Code Sample
Language/SDK

Both

主要言語
Python
スター
13.9k
フォーク
2.4k
平均マージ
1日 16時間
マージ済み PR（30日）
440

環境構築

Codespaces で開く

このプロジェクトの開発コンテナを、あなたの GitHub アカウントでブラウザ上に起動します。

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

microsoft/agent-framework のほかの issue

microsoft/agent-framework の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。