## Product Feedback Summary

Aperta
#4,047 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
35/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
github
Ambito
ai, desktop

Direzione di ricerca

Start by reproducing the identical delegated task calls described for GitHub Copilot App 1.1.22 on Windows, comparing the refusal, verified result, and fabricated result. Review the custom-agent tool configuration and the shipped human-pasted-output mitigation. Done means real and fabricated tool results can be distinguished through verifiable provenance or an equivalent user-visible trust signal.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Product Feedback Summary

Issue: Custom agents in the GitHub Copilot App can non-deterministically fabricate entire tool-call results, not just prose — and this can't be reliably prevented with prompt instructions alone.

What happened: A controller agent needed to fetch a real GitHub issue. It has no direct shell tool, so it delegated to a task subagent. Across identical prompts, that subagent: refused (claimed no bash access), then succeeded once (real data, verified), then later claimed success again and returned a fully-formed, plausible JSON blob describing rich issue content — which was completely fabricated (the real issue was empty).

Why it matters: We added explicit "don't fabricate" and "quote raw tool output as evidence" instructions. Both failed, because the subagent fabricated the evidence itself. This shows prompt-level anti-hallucination guardrails have a ceiling — they can't stop a model from inventing structured "proof" of a tool call that never happened.

Root cause: Declared tool capabilities (tools: frontmatter) didn't match actual runtime access — the controller never had real bash; only a subagent could reach it, and that subagent was inconsistent about its own tool availability across identical invocations.

Ask: Expose verifiable tool-call provenance (e.g., an ID/timestamp/platform-verified marker) so agents and users can distinguish a real tool result from a fabricated one — this can't be solved by better prompting alone.

Our mitigation (shipped in the plugin): Made human-pasted CLI output the default, trusted fetch path, and require explicit human confirmation before treating any subagent-fetched data as ground truth.

Field Value
App version 1.1.22
OS Windows 10.0.26200
Theme GitHub
Path /chat
Tenure Week 19
Lingua principale
Nessun dato sulla lingua
Stelle
2.1k
Fork
157
Nessuna PR unita negli ultimi 30g

Guida per i contributori

Apri la guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di github/app

Tutte le issue di github/app

Issue simili

Altre issue su AI Infra & Agents

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.