[bot] Bedrock Converse/ConverseStream requests: toolConfig (tool definitions) never captured in span metadata

Aperta Adatta ai principianti
#158 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
82/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Tranquilla
Stack tecnologico
aws, java
Ambito
api, backend

Direzione di ricerca

Inizia in braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java, in tagBedrockRequest, quindi esamina BraintrustBedrockInterceptor.java e BraintrustAWSBedrockTest.java. Aggiungi la copertura per la cattura delle richieste Converse e ConverseStream con toolConfig e verifica che le definizioni degli strumenti e il criterio di scelta dello strumento compaiano nei metadati acquisiti o in input_json.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Summary

InstrumentationSemConv.tagBedrockRequest() extracts inferenceConfig (maxTokens/temperature/topP/stopSequences), system, and messages from the Bedrock Converse/ConverseStream request body, but never reads the top-level toolConfig field. toolConfig carries the tool definitions and tool-choice policy sent to the model — a core, documented part of the Converse API request — and is silently dropped from every Bedrock LLM span's braintrust.metadata/braintrust.input_json, for both streaming and non-streaming calls.

This is a request-side gap and is distinct from issue #85 (which covers the response-side streaming reconstruction dropping tool-use content blocks in TeeingSubscriber/buildConverseJson). Even for non-streaming Converse calls where the response-side tool use is captured correctly by normalizeBedrockMessage(), the request never shows what tools were offered to the model or what tool-choice policy was set.

What is missing

In tagBedrockRequest() (InstrumentationSemConv.java, lines 339-396):

if (requestBody != null) {
    JsonNode requestJson = BraintrustJsonMapper.get().readTree(requestBody);
    if (requestJson.has("inferenceConfig")) { ... }   // handled
    if (requestJson.has("messages")) { ... }           // handled
}

There is no if (requestJson.has("toolConfig")) branch anywhere in this method. The Converse/ConverseStream request body's toolConfig object — containing tools (an array of toolSpec definitions with name/description/inputSchema) and toolChoice (auto/any/{tool: {name}}) — is read from requestBody but never surfaced into metadata or input_json. A trace of a tool-enabled Bedrock call shows the conversation and inference params but gives no visibility into which tools were available or how tool selection was constrained, even though the model's resulting stopReason: "tool_use" and tool call are otherwise correctly displayed.

Braintrust docs status

Upstream sources

Local files inspected

  • braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java — lines 339-396 (tagBedrockRequest: reads inferenceConfig, system, messages; no toolConfig handling anywhere in the method or file)
  • braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustBedrockInterceptor.java — passes the raw request body through to tagBedrockRequest; no separate toolConfig handling
  • braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/test/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustAWSBedrockTest.java — no test asserts on toolConfig appearing in captured metadata
Lingua principale
Java
Stelle
21
Fork
5
Merge medio
2g 7h
PR unite (30g)
8

Guida per i contributori

Apri la guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di braintrustdata/braintrust-sdk-java

Tutte le issue di braintrustdata/braintrust-sdk-java

Issue simili

Altre issue su Java

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.