[bot] AWS Bedrock InvokeModel for embeddings not instrumented
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 68/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- aws, java
- Domain
- cloud, observability
Research direction
Start with BraintrustBedrockInterceptor.java and BraintrustAWSBedrock.java to trace how Converse operations are selected, then read InstrumentationSemConv.java for the existing Bedrock request and response tagging. Review BraintrustAWSBedrockTest.java and add coverage for InvokeModelRequest using Titan and Cohere-shaped payloads. Done means embedding calls create spans with model/provider metadata, captured input, embedding type, and the specified token metrics.
Written by the indexing model from the issue text.
Description
Summary
The AWS Bedrock instrumentation in BraintrustBedrockInterceptor explicitly skips all operations other than Converse and ConverseStream. This means embedding model calls — which use the InvokeModel API, not Converse — produce no spans, no input capture, and no token metrics.
All embedding models on AWS Bedrock require InvokeModel:
- Amazon Titan Text Embeddings v2 (
amazon.titan-embed-text-v2:0) - Amazon Titan Text Embeddings v1 (
amazon.titan-embed-text-v1) - Amazon Titan Multimodal Embeddings (
amazon.titan-embed-image-v1) - Cohere Embed English v3 (
cohere.embed-english-v3) - Cohere Embed Multilingual v3 (
cohere.embed-multilingual-v3)
None of these models support the Converse API — InvokeModel is the only invocation path for Bedrock embeddings.
What is missing
BraintrustBedrockInterceptor.beforeExecution() (lines 60–71) explicitly limits instrumentation to two operations:
private static final Set<String> INSTRUMENTED_OPERATIONS = Set.of("Converse", "ConverseStream");
// Only instrument Converse and ConverseStream — other Bedrock operations
// (InvokeModel, ApplyGuardrail, etc.) are not LLM calls we know how to tag.
if (!INSTRUMENTED_OPERATIONS.contains(operationName)) {
return;
}
When a user calls client.invokeModel(...) with an embedding model (e.g. amazon.titan-embed-text-v2:0), the interceptor returns immediately without creating a span. No span is created, no input text is captured, and no embedding output or token metrics are recorded.
The BraintrustAWSBedrock.wrap() Javadoc on the sync builder explicitly states it traces "every converse call" and the async builder traces "every converseStream call" — embedding calls via InvokeModel are not in scope at all today.
A minimal InvokeModel embedding span should capture:
braintrust.metadata: model ID (from the request URI path/model/{modelId}/invoke), providerbraintrust.input_json: theinputText(or equivalent input field) from the request bodybraintrust.metrics:prompt_tokensfrom the responseinputTextTokenCount(Titan) ormeta.inputTokenCount(Cohere)braintrust.span_attributes:{"type": "embedding"}
Braintrust docs status
- AWS Bedrock listed as a supported cloud provider at https://www.braintrust.dev/docs/integrations/ai-providers: supported (for generative use)
- No mention of Bedrock embeddings or
InvokeModelinstrumentation in any Braintrust docs page: not_found - For comparison, OpenAI and Google GenAI embedding instrumentation exists in this repo (though with detail gaps tracked in #66 and #65)
Upstream sources
- AWS Bedrock InvokeModel API reference: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html — "You use model inference to generate text, images, and embeddings."
- Amazon Titan Text Embeddings v2 example: https://docs.aws.amazon.com/bedrock/latest/userguide/bedrock-runtime_example_bedrock-runtime_InvokeModel_TitanTextEmbeddings_section.html — uses
client.invokeModel()with model IDamazon.titan-embed-text-v2:0; response includesembeddingarray andinputTextTokenCount - Cohere Embed on Bedrock: https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-embed.html — uses
InvokeModelwithtextsarray; response includesembeddingsandmeta.inputTokenCount - AWS SDK for Java v2 BedrockRuntimeClient:
invokeModel(InvokeModelRequest)— operation nameInvokeModel, same SDK client as Converse
Local files inspected
braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustBedrockInterceptor.java— line 60:Set.of("Converse", "ConverseStream"); lines 63–71:beforeExecutionreturns without span for any other operation name includingInvokeModelbraintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustAWSBedrock.java— Javadoc on bothwrap()overloads confirms only Converse/ConverseStream are traced; noinvokeModelwrapping existsbraintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/test/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustAWSBedrockTest.java— all tests useConverseRequestandConverseStreamRequest; noInvokeModelRequesttest existsbraintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java—tagBedrockRequest()andtagBedrockResponse()only handle Converse-shaped JSON (messages, output.message, usage.inputTokens); InvokeModel request/response bodies have entirely different schemas
- Dominant language
- Java
- Stars
- 21
- Forks
- 5
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 8
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from braintrustdata/braintrust-sdk-java
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
All issues in braintrustdata/braintrust-sdk-java
Similar issues
-
certification
Difficulty 1/5 Under an hour Newbie friendliness 80/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
[BUG] ECR GetAuthorizationToken returns a proxyEndpoint for the default region, not the request's Openbug ecr
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Needs: Triage Type: Feature request
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
AntennaPod/AntennaPod#8794 ·
-
agentic-workflows
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
github/copilot-sdk#2760 ·