executeTool rejects top-level intent that its sibling callMcpTool accepts

Aperta
  • Qualcosa non va?
#2,455 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
48/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
typescript
Ambito
backend-api-design, tooling

Direzione di ricerca

Read packages/junior/src/chat/tool-support/catalog-tool-call.ts and compare the input contracts in packages/junior/src/chat/tools/execute-tool.ts and chat/tools/skill/call-mcp-tool.ts. Then locate the existing primary test for catalog tool calls and check how it covers top-level intent and parameters outside arguments. Done means the dispatcher envelope is documented consistently or the difference is explained, the test covers the chosen behavior, and scheduling eval runs show no rejected executeTool shapes.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Problem

The model often sends executeTool calls in a shape that the runtime rejects. Each rejection wastes one tool call. In the recurring-slot scheduling eval (#2413), that matters, because the case has a 60s idle limit and a tool-call budget.

Two shapes show up:

  1. A top-level intent key next to tool_name and arguments.
  2. Catalog tool parameters at the top level instead of under arguments.

Both are rejected by resolveCatalogToolCall() in packages/junior/src/chat/tool-support/catalog-tool-call.ts with executeTool arguments must be nested under arguments, not top-level fields: ....

Evidence

  • In about 98 recent live behavioral runs of the Google scheduling cases, 22 executeTool calls were rejected for their argument shape. The count is from an agent that read the eval transcripts on evals.sentry.dev. It is not re-counted here.
  • One example: run e3899820, recurring-slot case. The first google_findMeetingTimes call failed because the model put intent inside the executeTool input.

Likely cause (inference, not confirmed)

The two dispatcher tools have different input contracts:

  • callMcpTool (chat/tools/skill/call-mcp-tool.ts) accepts a top-level intent. Its extra-key check allows tool_name, arguments and intent.
  • searchMcpTools also takes a top-level intent.
  • executeTool (chat/tools/execute-tool.ts) is .strict() and allows only tool_name and arguments.

The model sees both dispatchers in the same turn. It is plausible that it carries intent over from callMcpTool to executeTool. This issue does not prove that; a check of which calls carried intent versus nested parameters would.

Options

  • Make the two dispatchers take the same envelope. Either executeTool accepts an optional intent (and passes it where it is useful, such as auth prompts), or neither does.
  • Keep executeTool strict, but make the rejection cheaper: the error could name the corrected call shape.

Do not change eval messages, checks or timeouts to hide this. Fix it in the tool contract.

Done when

  • The two dispatchers have one documented argument envelope, or the difference is deliberate and explained beside the code.
  • The existing primary test for catalog tool calls covers the chosen behavior.
  • A live behavioral run of the scheduling cases shows no rejected executeTool shapes.

Refs #2413.

via David Cramer.

--

View Session [Sentry]

Lingua principale
TypeScript
Stelle
385
Fork
42
Merge medio
9h 32m
PR unite (30g)
350

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di getsentry/junior

Tutte le issue di getsentry/junior

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.