[bot] Bedrock Converse/ConverseStream requests: toolConfig (tool definitions) never captured in span metadata
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 2/5
- Tempo stimato
- 1-3 ore
- Idoneità per principianti
- 82/100
Direzione di ricerca
Inizia in braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java, in tagBedrockRequest, quindi esamina BraintrustBedrockInterceptor.java e BraintrustAWSBedrockTest.java. Aggiungi la copertura per la cattura delle richieste Converse e ConverseStream con toolConfig e verifica che le definizioni degli strumenti e il criterio di scelta dello strumento compaiano nei metadati acquisiti o in input_json.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Summary
InstrumentationSemConv.tagBedrockRequest() extracts inferenceConfig (maxTokens/temperature/topP/stopSequences), system, and messages from the Bedrock Converse/ConverseStream request body, but never reads the top-level toolConfig field. toolConfig carries the tool definitions and tool-choice policy sent to the model — a core, documented part of the Converse API request — and is silently dropped from every Bedrock LLM span's braintrust.metadata/braintrust.input_json, for both streaming and non-streaming calls.
This is a request-side gap and is distinct from issue #85 (which covers the response-side streaming reconstruction dropping tool-use content blocks in TeeingSubscriber/buildConverseJson). Even for non-streaming Converse calls where the response-side tool use is captured correctly by normalizeBedrockMessage(), the request never shows what tools were offered to the model or what tool-choice policy was set.
What is missing
In tagBedrockRequest() (InstrumentationSemConv.java, lines 339-396):
if (requestBody != null) {
JsonNode requestJson = BraintrustJsonMapper.get().readTree(requestBody);
if (requestJson.has("inferenceConfig")) { ... } // handled
if (requestJson.has("messages")) { ... } // handled
}
There is no if (requestJson.has("toolConfig")) branch anywhere in this method. The Converse/ConverseStream request body's toolConfig object — containing tools (an array of toolSpec definitions with name/description/inputSchema) and toolChoice (auto/any/{tool: {name}}) — is read from requestBody but never surfaced into metadata or input_json. A trace of a tool-enabled Bedrock call shows the conversation and inference params but gives no visibility into which tools were available or how tool selection was constrained, even though the model's resulting stopReason: "tool_use" and tool call are otherwise correctly displayed.
Braintrust docs status
- Bedrock is listed as a supported provider at https://www.braintrust.dev/docs/integrations/ai-providers: supported
- No Bedrock-specific mention of tool/function-call request metadata capture on Braintrust docs: not_found
Upstream sources
- AWS Bedrock Converse API reference: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html — documents
toolConfigas a top-level request field alongsidemessages,system,inferenceConfig - AWS Bedrock ConverseStream API reference: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ConverseStream.html — same
toolConfigrequest field - AWS Bedrock tool use guide: https://docs.aws.amazon.com/bedrock/latest/userguide/tool-use.html — describes
toolConfig.tools/toolConfig.toolChoicesemantics
Local files inspected
braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java— lines 339-396 (tagBedrockRequest: readsinferenceConfig,system,messages; notoolConfighandling anywhere in the method or file)braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustBedrockInterceptor.java— passes the raw request body through totagBedrockRequest; no separate toolConfig handlingbraintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/test/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustAWSBedrockTest.java— no test asserts ontoolConfigappearing in captured metadata
- Lingua principale
- Java
- Stelle
- 21
- Fork
- 5
- Merge medio
- 2g 7h
- PR unite (30g)
- 8
Guida per i contributori
Apri la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di braintrustdata/braintrust-sdk-java
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
-
Difficoltà 5/5 Più di una settimana Idoneità per principianti 35/100
Tutte le issue di braintrustdata/braintrust-sdk-java
Issue simili
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
infinispan/infinispan#18150 ·
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 84/100
-
untriaged
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
opensearch-project/k-NN#3597 ·
-
bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 88/100
-
bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100