## Product Feedback Summary

Start by reproducing the identical delegated task calls described for GitHub Copilot App 1.1.22 on Windows, comparing the refusal, verified result, and fabricated result. Review the custom-agent tool configuration and the shipped human-pasted-output mitigation. Done means real and fabricated tool results can be distinguished through verifiable provenance or an equivalent user-visible trust signal.

Product Feedback Summary

Issue: Custom agents in the GitHub Copilot App can non-deterministically fabricate entire tool-call results, not just prose — and this can't be reliably prevented with prompt instructions alone.

What happened: A controller agent needed to fetch a real GitHub issue. It has no direct shell tool, so it delegated to a task subagent. Across identical prompts, that subagent: refused (claimed no bash access), then succeeded once (real data, verified), then later claimed success again and returned a fully-formed, plausible JSON blob describing rich issue content — which was completely fabricated (the real issue was empty).

Why it matters: We added explicit "don't fabricate" and "quote raw tool output as evidence" instructions. Both failed, because the subagent fabricated the evidence itself. This shows prompt-level anti-hallucination guardrails have a ceiling — they can't stop a model from inventing structured "proof" of a tool call that never happened.

Root cause: Declared tool capabilities (tools: frontmatter) didn't match actual runtime access — the controller never had real bash; only a subagent could reach it, and that subagent was inconsistent about its own tool availability across identical invocations.

Ask: Expose verifiable tool-call provenance (e.g., an ID/timestamp/platform-verified marker) so agents and users can distinguish a real tool result from a fabricated one — this can't be solved by better prompting alone.

Our mitigation (shipped in the plugin): Made human-pasted CLI output the default, trusted fetch path, and require explicit human confirmation before treating any subagent-fetched data as ground truth.