[bot] Bedrock Converse/ConverseStream requests: toolConfig (tool definitions) never captured in span metadata
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 82/100
Research direction
Start in braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java at tagBedrockRequest, then inspect BraintrustBedrockInterceptor.java and BraintrustAWSBedrockTest.java. Add coverage for toolConfig on Converse and ConverseStream request capture, and verify the tool definitions and tool-choice policy appear in the captured metadata or input_json.
Written by the indexing model from the issue text.
Description
Summary
InstrumentationSemConv.tagBedrockRequest() extracts inferenceConfig (maxTokens/temperature/topP/stopSequences), system, and messages from the Bedrock Converse/ConverseStream request body, but never reads the top-level toolConfig field. toolConfig carries the tool definitions and tool-choice policy sent to the model — a core, documented part of the Converse API request — and is silently dropped from every Bedrock LLM span's braintrust.metadata/braintrust.input_json, for both streaming and non-streaming calls.
This is a request-side gap and is distinct from issue #85 (which covers the response-side streaming reconstruction dropping tool-use content blocks in TeeingSubscriber/buildConverseJson). Even for non-streaming Converse calls where the response-side tool use is captured correctly by normalizeBedrockMessage(), the request never shows what tools were offered to the model or what tool-choice policy was set.
What is missing
In tagBedrockRequest() (InstrumentationSemConv.java, lines 339-396):
if (requestBody != null) {
JsonNode requestJson = BraintrustJsonMapper.get().readTree(requestBody);
if (requestJson.has("inferenceConfig")) { ... } // handled
if (requestJson.has("messages")) { ... } // handled
}
There is no if (requestJson.has("toolConfig")) branch anywhere in this method. The Converse/ConverseStream request body's toolConfig object — containing tools (an array of toolSpec definitions with name/description/inputSchema) and toolChoice (auto/any/{tool: {name}}) — is read from requestBody but never surfaced into metadata or input_json. A trace of a tool-enabled Bedrock call shows the conversation and inference params but gives no visibility into which tools were available or how tool selection was constrained, even though the model's resulting stopReason: "tool_use" and tool call are otherwise correctly displayed.
Braintrust docs status
- Bedrock is listed as a supported provider at https://www.braintrust.dev/docs/integrations/ai-providers: supported
- No Bedrock-specific mention of tool/function-call request metadata capture on Braintrust docs: not_found
Upstream sources
- AWS Bedrock Converse API reference: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html — documents
toolConfigas a top-level request field alongsidemessages,system,inferenceConfig - AWS Bedrock ConverseStream API reference: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ConverseStream.html — same
toolConfigrequest field - AWS Bedrock tool use guide: https://docs.aws.amazon.com/bedrock/latest/userguide/tool-use.html — describes
toolConfig.tools/toolConfig.toolChoicesemantics
Local files inspected
braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java— lines 339-396 (tagBedrockRequest: readsinferenceConfig,system,messages; notoolConfighandling anywhere in the method or file)braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustBedrockInterceptor.java— passes the raw request body through totagBedrockRequest; no separate toolConfig handlingbraintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/test/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustAWSBedrockTest.java— no test asserts ontoolConfigappearing in captured metadata
- Dominant language
- Java
- Stars
- 21
- Forks
- 5
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 8
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from braintrustdata/braintrust-sdk-java
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
All issues in braintrustdata/braintrust-sdk-java
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
infinispan/infinispan#18150 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
untriaged
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
opensearch-project/k-NN#3597 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100