## Product Feedback Summary

Abierto
#4,047 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
35/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
github
Área
ai, desktop

Línea de trabajo

Start by reproducing the identical delegated task calls described for GitHub Copilot App 1.1.22 on Windows, comparing the refusal, verified result, and fabricated result. Review the custom-agent tool configuration and the shipped human-pasted-output mitigation. Done means real and fabricated tool results can be distinguished through verifiable provenance or an equivalent user-visible trust signal.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Product Feedback Summary

Issue: Custom agents in the GitHub Copilot App can non-deterministically fabricate entire tool-call results, not just prose — and this can't be reliably prevented with prompt instructions alone.

What happened: A controller agent needed to fetch a real GitHub issue. It has no direct shell tool, so it delegated to a task subagent. Across identical prompts, that subagent: refused (claimed no bash access), then succeeded once (real data, verified), then later claimed success again and returned a fully-formed, plausible JSON blob describing rich issue content — which was completely fabricated (the real issue was empty).

Why it matters: We added explicit "don't fabricate" and "quote raw tool output as evidence" instructions. Both failed, because the subagent fabricated the evidence itself. This shows prompt-level anti-hallucination guardrails have a ceiling — they can't stop a model from inventing structured "proof" of a tool call that never happened.

Root cause: Declared tool capabilities (tools: frontmatter) didn't match actual runtime access — the controller never had real bash; only a subagent could reach it, and that subagent was inconsistent about its own tool availability across identical invocations.

Ask: Expose verifiable tool-call provenance (e.g., an ID/timestamp/platform-verified marker) so agents and users can distinguish a real tool result from a fabricated one — this can't be solved by better prompting alone.

Our mitigation (shipped in the plugin): Made human-pasted CLI output the default, trusted fetch path, and require explicit human confirmation before treating any subagent-fetched data as ground truth.

Field Value
App version 1.1.22
OS Windows 10.0.26200
Theme GitHub
Path /chat
Tenure Week 19
Lenguaje dominante
Sin datos de lenguaje
Estrellas
2.1k
Forks
157
Sin PR fusionados en 30 d

Guía de contribución

Abrir la guía de contribución

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de github/app

Todos los issues de github/app

Issues similares

Más issues de AI Infra & Agents

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.