Redaction placeholder `[hidden]` is confusable with ARIA hidden semantics; agent misreads it as a site defect

Open Beginner friendly
#613 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
68/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Quiet
Tech stack
typescript
Domain
ai, security

Research direction

Start in packages/core/src/security/actionFirewall.ts at redactSensitiveValues and trace where the [hidden] replacement is produced and exposed to the agent. Replace the ambiguous token with a self-describing redaction placeholder, then check the relevant snapshots or validation output for consistent replacement text. If the system prompt is updated too, confirm it explains that the value was withheld by the caller rather than hidden on the page.

Written by the indexing model from the issue text.

Description

Minor follow-up to #607 / #608.

The redaction placeholder [hidden] (redactSensitiveValues in packages/core/src/security/actionFirewall.ts) collides with the vocabulary the agent already uses for accessibility-tree semantics. The model reads a redacted field as an element that is hidden on the page rather than a value that was deliberately withheld from it, and then reasons about it as a site defect.

Observed in a production smoke test of #608 — task was to fill a field from caller data and report the resulting field text:

The 'Customer name' field was successfully filled as confirmed by the fill_user_data tool. However, I am unable to provide the exact verbatim text that appears in the 'Customer name' field because the field (E4) is marked as [hidden] in the accessibility tree. As an agent, my perception of the page content is solely based on this tree, and hidden elements are not accessible for visual confirmation or direct extraction of their displayed text. This represents a genuine site limitation preventing the fulfillment of the verbatim text reporting requirement.

The security behaviour is correct — the value never reached the model. The cost is wasted work and a misleading final answer: 4 iterations and 2 validation errors before the run ended force-accepted, with the agent attributing the redaction to the website.

A self-describing token would avoid sending the agent down that path, e.g. [redacted-caller-data] or [value-withheld]. Cheap change; the only consideration is that it lengthens the replacement string slightly in snapshots where many values are redacted.

Possibly worth pairing with a line in the system prompt explaining that [...] in a field value means the caller supplied it and the agent is not permitted to read it back — so the agent treats it as expected rather than as a page defect worth retrying.

Dominant language
TypeScript
Stars
100
Forks
14
Avg merge
15d 7h
Merged PRs (30d)
13

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from mozilla/pilo

All issues in mozilla/pilo

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.