Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Follow-up to #5330: recommended guardrail pattern verified on 0.23.1 (results + one output_info papercut)

Cerrado
  • ¿Algo no está bien?
#5,351 2 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
2/5
Tiempo estimado
1-3 horas
Aptitud para principiantes
55/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
python
Área
ai-infra-agents, tooling

Línea de trabajo

Read the ToolGuardrailFunctionOutput.raise_exception() implementation in agents/tool_guardrails.py and compare it with reject_content(). Reproduce the reported behavior by passing output_info to both methods and inspecting their behavior values; done means the hard-stop path either preserves the reason or warns that the argument is unused. The issue does not name a test file, so locate the relevant guardrail tests before adding coverage.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

cc @jbeckwith-oai — following up from #5330. The results were also posted as a comment on the closed thread for the record; reposting here since a closed thread is easy to miss. You suggested the shared-context pattern and the serial configuration — we implemented both on 0.23.1 and verified everything. Results below, plus one small papercut.

  1. Shared state via Runner.run(..., context=state) — confirmed. The plan, halt flag, and observed-call log persist across tool calls and handoffs via data.context.context; no globals needed.

  2. Hard-stop / soft-reject semantics — confirmed, both finer points:

  • Out-of-plan call → raise_exception() → run stops with ToolInputGuardrailTripwireTriggered.
  • Soft-reject: the out-of-plan call sets the halt flag and returns reject_content(); the run completes and the model reports which calls succeeded and which were blocked. Later calls (even in-plan ones) are rejected by the halt-flag check while the model wraps up.
  • Fresh-object-per-run isolation confirmed: a second run with a fresh state object runs clean after a first run halted; reusing the halted state object keeps blocking.
  1. The race window, quantified. We injected 6 tool calls (4 in-plan, 2 out-of-plan) directly into the batch executor Runner.run uses internally — no LLM, n=30 per configuration:
  • max_function_tool_concurrency=1: execution order matched submission order, zero timeline overlap in 30/30 runs; at tripwire time only 3/6 calls had started — the window cannot open under serial execution.
  • Default concurrent setting: timeline overlap in 30/30 runs; all 6 calls had already started at tripwire time — the window does reproduce.
  • Worst case: 5 high-risk out-of-plan calls (money transfer, file deletion, log purge, bank wire, DB drop) → 5/5 vetoed with zero side-effect leakage in both serial and concurrent settings (n=30 each).

  1. One papercut for anyone implementing this: a hard-stop tripwire can't carry its reason —

    from agents.tool_guardrails import ToolGuardrailFunctionOutput

    out = ToolGuardrailFunctionOutput.raise_exception(output_info={"rule": "R1_plan_breach"})
    print(out.behavior) # {'type': 'raise_exception'} — output_info dropped

    ok = ToolGuardrailFunctionOutput.reject_content(message="blocked", output_info={"rule": "R1"})
    print(ok.behavior) # {'type': 'reject_content', 'message': 'blocked'} — message kept

We pass the veto reason through the shared context object instead; either surfacing output_info in the behavior, or warning on the unused kwarg, would fix it.

  1. Coverage boundary (documented explicitly on our side): tool input guardrails don't intercept hosted tools or handoffs — coverage claims are scoped to local function tools.

Environment: openai-agents 0.23.1 (latest release), Python 3.12.13, Debian 13. Repro scripts and full result artifacts: https://github.com/lsy1533133/phm-agent-verifier-tests

Lenguaje dominante
Python
Estrellas
29.9k
Forks
4.8k
Merge medio
22 h 17 min
PR fusionados (30 d)
208

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de openai/openai-agents-python

Todos los issues de openai/openai-agents-python

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.