Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Follow-up to #5330: recommended guardrail pattern verified on 0.23.1 (results + one output_info papercut)

Chiusa
  • Qualcosa non va?
#5,351 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
55/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
python
Ambito
ai-infra-agents, tooling

Direzione di ricerca

Read the ToolGuardrailFunctionOutput.raise_exception() implementation in agents/tool_guardrails.py and compare it with reject_content(). Reproduce the reported behavior by passing output_info to both methods and inspecting their behavior values; done means the hard-stop path either preserves the reason or warns that the argument is unused. The issue does not name a test file, so locate the relevant guardrail tests before adding coverage.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

cc @jbeckwith-oai — following up from #5330. The results were also posted as a comment on the closed thread for the record; reposting here since a closed thread is easy to miss. You suggested the shared-context pattern and the serial configuration — we implemented both on 0.23.1 and verified everything. Results below, plus one small papercut.

  1. Shared state via Runner.run(..., context=state) — confirmed. The plan, halt flag, and observed-call log persist across tool calls and handoffs via data.context.context; no globals needed.

  2. Hard-stop / soft-reject semantics — confirmed, both finer points:

  • Out-of-plan call → raise_exception() → run stops with ToolInputGuardrailTripwireTriggered.
  • Soft-reject: the out-of-plan call sets the halt flag and returns reject_content(); the run completes and the model reports which calls succeeded and which were blocked. Later calls (even in-plan ones) are rejected by the halt-flag check while the model wraps up.
  • Fresh-object-per-run isolation confirmed: a second run with a fresh state object runs clean after a first run halted; reusing the halted state object keeps blocking.
  1. The race window, quantified. We injected 6 tool calls (4 in-plan, 2 out-of-plan) directly into the batch executor Runner.run uses internally — no LLM, n=30 per configuration:
  • max_function_tool_concurrency=1: execution order matched submission order, zero timeline overlap in 30/30 runs; at tripwire time only 3/6 calls had started — the window cannot open under serial execution.
  • Default concurrent setting: timeline overlap in 30/30 runs; all 6 calls had already started at tripwire time — the window does reproduce.
  • Worst case: 5 high-risk out-of-plan calls (money transfer, file deletion, log purge, bank wire, DB drop) → 5/5 vetoed with zero side-effect leakage in both serial and concurrent settings (n=30 each).

  1. One papercut for anyone implementing this: a hard-stop tripwire can't carry its reason —

    from agents.tool_guardrails import ToolGuardrailFunctionOutput

    out = ToolGuardrailFunctionOutput.raise_exception(output_info={"rule": "R1_plan_breach"})
    print(out.behavior) # {'type': 'raise_exception'} — output_info dropped

    ok = ToolGuardrailFunctionOutput.reject_content(message="blocked", output_info={"rule": "R1"})
    print(ok.behavior) # {'type': 'reject_content', 'message': 'blocked'} — message kept

We pass the veto reason through the shared context object instead; either surfacing output_info in the behavior, or warning on the unused kwarg, would fix it.

  1. Coverage boundary (documented explicitly on our side): tool input guardrails don't intercept hosted tools or handoffs — coverage claims are scoped to local function tools.

Environment: openai-agents 0.23.1 (latest release), Python 3.12.13, Debian 13. Repro scripts and full result artifacts: https://github.com/lsy1533133/phm-agent-verifier-tests

Lingua principale
Python
Stelle
30k
Fork
4.9k
Merge medio
15h 43m
PR unite (30g)
211

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di openai/openai-agents-python

Tutte le issue di openai/openai-agents-python

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.