## Product Feedback Summary
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
Research direction
Start by reproducing the identical delegated task calls described for GitHub Copilot App 1.1.22 on Windows, comparing the refusal, verified result, and fabricated result. Review the custom-agent tool configuration and the shipped human-pasted-output mitigation. Done means real and fabricated tool results can be distinguished through verifiable provenance or an equivalent user-visible trust signal.
Written by the indexing model from the issue text.
Description
Product Feedback Summary
Issue: Custom agents in the GitHub Copilot App can non-deterministically fabricate entire tool-call results, not just prose — and this can't be reliably prevented with prompt instructions alone.
What happened: A controller agent needed to fetch a real GitHub issue. It has no direct shell tool, so it delegated to a task subagent. Across identical prompts, that subagent: refused (claimed no bash access), then succeeded once (real data, verified), then later claimed success again and returned a fully-formed, plausible JSON blob describing rich issue content — which was completely fabricated (the real issue was empty).
Why it matters: We added explicit "don't fabricate" and "quote raw tool output as evidence" instructions. Both failed, because the subagent fabricated the evidence itself. This shows prompt-level anti-hallucination guardrails have a ceiling — they can't stop a model from inventing structured "proof" of a tool call that never happened.
Root cause: Declared tool capabilities (tools: frontmatter) didn't match actual runtime access — the controller never had real bash; only a subagent could reach it, and that subagent was inconsistent about its own tool availability across identical invocations.
Ask: Expose verifiable tool-call provenance (e.g., an ID/timestamp/platform-verified marker) so agents and users can distinguish a real tool result from a fabricated one — this can't be solved by better prompting alone.
Our mitigation (shipped in the plugin): Made human-pasted CLI output the default, trusted fetch path, and require explicit human confirmation before treating any subagent-fetched data as ground truth.
| Field | Value |
|---|---|
| App version | 1.1.22 |
| OS | Windows 10.0.26200 |
| Theme | GitHub |
| Path | /chat |
| Tenure | Week 19 |
- Dominant language
- No language data
- Stars
- 2.1k
- Forks
- 157
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from github/app
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 64/100
Similar issues
-
triage/confirmed
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
agentscope-ai/agentscope#2775 ·
-
area/sessions comp/cron comp/gateway P2 sweeper:risk-message-delivery sweeper:risk-session-state type/bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
NousResearch/hermes-agent#118863 ·
-
automation missing-model model-sync provider:pioneer
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
anomalyco/models.dev#7701 ·
-
external-plugin ready-for-review
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
github/awesome-copilot#3586 · 2 comments ·
-
bug-unconfirmed
Difficulty 2/5 1-3 hours Newbie friendliness 76/100