Risk Agent help text promises a "show it" control that no longer exists
Nobody has claimed this yet.
Assessment
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Newbie friendliness
- 76/100
- Issue type
- Documentation
- Clarity
- Mostly clear
- Activity status
- Active
- Domain
- documentation
Research direction
Start in the agent Runtime settings UI and locate the Output Guardrail description under Guardrails. Update the help text to match the current verdict behavior, including that flagged answers are replaced in chat and remain available in the trace. Reproduce the listed steps to verify the rendered description no longer promises a show control.
Written by the indexing model from the issue text.
Description
Describe the bug
The output-guardrail help text in the agent Runtime settings tells the user a flagged answer can be revealed on demand:
"Checks the agent's answer once it is complete. A flagged answer is withheld in chat until you choose to show it, alongside the detected risks."
That control was deliberately removed. The change that shipped the current verdict UI states:
"A flagged answer is now withheld outright: 'Show answer' is gone, so the verdict replaces the answer rather than hiding it behind a toggle. The answer text itself is untouched and still readable in the trace."
So the settings panel describes behaviour the product no longer has. A user configuring an output guard would reasonably expect to be able to inspect a withheld answer in the chat, and there is no way to do so.
Impact. Small but specific: it sets a wrong expectation at exactly the moment someone is deciding whether to turn the guard on. Anyone choosing Risk Agent partly because they believed they could still review flagged answers in place has been misled about what they are enabling. The answer does remain readable in the trace, so the information is not lost — but that is a different surface with different access, and the help text does not say so.
Not affected: the guard itself, which runs and reports correctly.
Steps to reproduce the behavior
1. Open any pydantic_ai agent → Runtime tab → Edit.
2. Scroll to the Guardrails section.
3. Set Output Guardrail to "Risk Agent".
4. Read the description rendered under the dropdown.
Observed: "A flagged answer is withheld in chat until you choose to show it, alongside
the detected risks."
There is no control in the chat view to show a withheld answer.
Expected Behavior
The help text matches the shipped behaviour — a flagged answer is withheld and replaced by the verdict and its risk categories, with the full text available in the trace rather than in the chat.
Relevant log output
Environment
- Env: dev (dev.design.akd.odsi.io)
- Browser: Chrome
- Date observed: 2026-08-25
No errors. This is a text/behaviour mismatch in the settings panel, not a runtime fault.
Found while running the output guardrail end to end: the guard was verified to run
(verdict recorded on every turn, provider "RiskAgent", latency ~5.7s) and to pass content
correctly. Only the description is wrong.
- Dominant language
- Jupyter Notebook
- Stars
- 4
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from NASA-IMPACT/akd-core
-
Difficulty 4/5 3-5 days Newbie friendliness 45/100
NASA-IMPACT/akd-core#569 ·
-
bug
NASA-IMPACT/akd-core#568 · 3 assignees ·
-
NASA-IMPACT/akd-core#567 · 2 comments · 1 assignee ·
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
NASA-IMPACT/akd-core#566 ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
NASA-IMPACT/akd-core#565 ·
All issues in NASA-IMPACT/akd-core
Similar issues
-
user-reported
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Kong/developer.konghq.com#7316 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
HarperFast/skills#96 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
infinispan/infinispan#18150 ·
-
bug triage:deciding
Difficulty 1/5 Under an hour Newbie friendliness 88/100
open-telemetry/otel-arrow#4132 ·
-
Ecosystem: ClawMetry — the Qwen Code reader is now free and open source (follow-up to #9294 / #9338) Opencategory/integration priority/P3 scope/documentation status/ready-for-human type/feature-request
Difficulty 1/5 Under an hour Newbie friendliness 84/100