Agent Persona Exploration - 2026-11-04

Closed Beginner friendly
  • Is something wrong?
#74,891 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
72/100
Issue type
Documentation
Clarity
Clearly specified
Activity status
Active
Tech stack
github, github-actions
Domain
devtools, documentation

Research direction

Read .github/aw/create-agentic-workflow-trigger-details.md and update the compliance-audit example described in the issue: change its permission and add the network setting. Then review .github/aw/report.md and .github/aw/create-agentic-workflow.md for the two additional recommendations. Done when the examples and guidance match the issue’s specified least-privilege and consistency rules.

Written by the indexing model from the issue text.

Description

agent-research
Persona Overview
  • Agent: agentic-workflows (via the persona-evaluator Claude Code sub-agent)
  • Personas This Run: Program Manager, Designer, Legal / Compliance (one-time baseline)
  • Scenarios Tested: 4 (PM-1, DS-1, LC-1, LC-2). All 4 invocations succeeded.
  • Average Quality Score: 4.3/5.0
Key Findings
  • The trigger was right in every case. The scheduled scenarios used a fuzzy weekly schedule plus workflow_dispatch. The PR scenarios used pull_request with paths: filters. DS-1 also added edited, so the review re-runs when the author adds a design link.
  • Permissions stayed read-only in all 4. Every write went through safe outputs. LC-2 even rejected the issues: write shown in the guide's own compliance-audit example.
  • Network scoping is inconsistent. DS-1 used network.allowed: []. PM-1 and LC-2 used network: defaults, even though they only read repository files and GitHub data. create-agentic-workflow.md §5 prescribes allowed: [] for exactly the LC-2 case.
  • Every response stated its assumptions. Each one named an ambiguity, such as what "overdue dependency" means, where the policy file lives, or where the design link is expected, and gave a fallback.
  • The prose and the frontmatter sketch sometimes disagree. In PM-1 the prose says to comment on an existing issue, but add-comment isn't configured. In LC-1 the prose lists more lockfile paths than the sketch. In LC-2 the prose says gh-proxy, but the sketch doesn't set that mode.
Top Patterns
  1. Triggers: fuzzy schedule + workflow_dispatch (2 of 4); pull_request + paths: (2 of 4)
  2. Tools: github in all 4 (with gh-proxy in 2). bash restricted to read-only commands in 2 of 4. No web or browser tools were suggested.
  3. Safe outputs: create-issue with close-older-issues for the scheduled reports. add-comment with hide-older-comments, plus an add-labels allowlist, for PR reviews. noop was always defined.
  4. Security: least-privilege read permissions in all 4. pull_request is preferred over pull_request_target. LC-1 explicitly said to treat package metadata as untrusted data.
View Scores by Scenario
ID Persona Trigger Tools Security Prompt Complete Avg
PM-1 Program Manager 5 4 3 4 4 4.0
DS-1 Designer 5 5 5 5 4 4.8
LC-1 Legal / Compliance 5 4 4 5 4 4.4
LC-2 Legal / Compliance 5 4 3 4 4 4.0
View High Quality Responses (Top 2)

  • DS-1, design token review (4.8):

    • Uses the edited trigger type so the review re-runs when a link is added.
    • Sets network.allowed: [], and checks design links only for presence and format, without fetching them.
    • Applies concrete WCAG 2.2 contrast thresholds (4.5:1 and 3:1).
    • Lists the components affected by renamed tokens.
    • Restricts labels with add-labels allowed:.
    • Posts one rolling comment, using hide-older-comments.

  • LC-1, SPDX license gate (4.4):

    • Sorts licenses into allowed, disallowed and uncertain. NOASSERTION, custom licenses and uncovered OR/AND/WITH expressions go to human review.
    • Has an explicit "never guess a license" rule.
    • Keeps the workflow advisory (no blocking review) and offers create-check-run as an option for a hard gate.
View Areas for Improvement (Top 3)

  • Network defaults on audits that only read the repository (PM-1, LC-2):

    • Both used network: defaults even though workflow-constraints.md:48 and create-agentic-workflow.md:211 say to use allowed: [].
    • Likely cause: the scheduled compliance-audit example in create-agentic-workflow-trigger-details.md (around lines 109–124) has no network: block and grants issues: write.

  • Ecosystems not inferred from the repository (LC-1): LC-1 enabled six ecosystems (node, python, go, rust, java, ruby) up front. The §5 guidance says to infer them from the manifests.

  • Prose and sketch drift:

    • PM-1's dedup plan says to comment on an existing issue, but there is no add-comment safe output. Its weekly dedup key also overlaps with close-older-issues.
    • LC-2 uses a monthly dedup key on a weekly schedule.
    • LC-1's bash allowlist uses "git diff:*", which differs from the documented "cmd *" and bare-command forms.
Recommendations
  1. Fix the compliance-audit example in .github/aw/create-agentic-workflow-trigger-details.md. Change issues: write to issues: read and add network: { allowed: [] }. The agent appears to copy this sample, and today it contradicts create-agentic-workflow.md §5 and the strict-mode rules.
  2. Add a scheduled-report network rule to .github/aw/report.md. State that digests and audits that only read GitHub through gh-proxy should use network.allowed: [], not defaults. PM-1 shows the gap for non-compliance digests.
  3. Add a consistency check to the ad hoc evaluation checklist in .github/aw/create-agentic-workflow.md. Every action the prompt describes (for example "comment on the existing issue") needs a matching configured safe output. The dedup-key granularity should match the schedule cadence. Ecosystems should be inferred from the manifests that actually exist.

References:

Generated by 🎭 Agent Persona Explorer · claude · routed: opus55 high · ~128 AIC · ⌖ ~2 AIC · ⊞ 6.6K · ◷

Dominant language
Go
Stars
5.6k
Forks
615
Avg merge
5h 36m
Merged PRs (30d)
805

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from github/gh-aw

All issues in github/gh-aw

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.