Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

[bot] AWS Bedrock InvokeModel for embeddings not instrumented

Open
#134 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
68/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Quiet
Tech stack
aws, java

Research direction

Start with BraintrustBedrockInterceptor.java and BraintrustAWSBedrock.java to trace how Converse operations are selected, then read InstrumentationSemConv.java for the existing Bedrock request and response tagging. Review BraintrustAWSBedrockTest.java and add coverage for InvokeModelRequest using Titan and Cohere-shaped payloads. Done means embedding calls create spans with model/provider metadata, captured input, embedding type, and the specified token metrics.

Written by the indexing model from the issue text.

Description

Summary

The AWS Bedrock instrumentation in BraintrustBedrockInterceptor explicitly skips all operations other than Converse and ConverseStream. This means embedding model calls — which use the InvokeModel API, not Converse — produce no spans, no input capture, and no token metrics.

All embedding models on AWS Bedrock require InvokeModel:

  • Amazon Titan Text Embeddings v2 (amazon.titan-embed-text-v2:0)
  • Amazon Titan Text Embeddings v1 (amazon.titan-embed-text-v1)
  • Amazon Titan Multimodal Embeddings (amazon.titan-embed-image-v1)
  • Cohere Embed English v3 (cohere.embed-english-v3)
  • Cohere Embed Multilingual v3 (cohere.embed-multilingual-v3)

None of these models support the Converse API — InvokeModel is the only invocation path for Bedrock embeddings.

What is missing

BraintrustBedrockInterceptor.beforeExecution() (lines 60–71) explicitly limits instrumentation to two operations:

private static final Set<String> INSTRUMENTED_OPERATIONS = Set.of("Converse", "ConverseStream");

// Only instrument Converse and ConverseStream — other Bedrock operations
// (InvokeModel, ApplyGuardrail, etc.) are not LLM calls we know how to tag.
if (!INSTRUMENTED_OPERATIONS.contains(operationName)) {
    return;
}

When a user calls client.invokeModel(...) with an embedding model (e.g. amazon.titan-embed-text-v2:0), the interceptor returns immediately without creating a span. No span is created, no input text is captured, and no embedding output or token metrics are recorded.

The BraintrustAWSBedrock.wrap() Javadoc on the sync builder explicitly states it traces "every converse call" and the async builder traces "every converseStream call" — embedding calls via InvokeModel are not in scope at all today.

A minimal InvokeModel embedding span should capture:

  • braintrust.metadata: model ID (from the request URI path /model/{modelId}/invoke), provider
  • braintrust.input_json: the inputText (or equivalent input field) from the request body
  • braintrust.metrics: prompt_tokens from the response inputTextTokenCount (Titan) or meta.inputTokenCount (Cohere)
  • braintrust.span_attributes: {"type": "embedding"}

Braintrust docs status

  • AWS Bedrock listed as a supported cloud provider at https://www.braintrust.dev/docs/integrations/ai-providers: supported (for generative use)
  • No mention of Bedrock embeddings or InvokeModel instrumentation in any Braintrust docs page: not_found
  • For comparison, OpenAI and Google GenAI embedding instrumentation exists in this repo (though with detail gaps tracked in #66 and #65)

Upstream sources

Local files inspected

  • braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustBedrockInterceptor.java — line 60: Set.of("Converse", "ConverseStream"); lines 63–71: beforeExecution returns without span for any other operation name including InvokeModel
  • braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustAWSBedrock.java — Javadoc on both wrap() overloads confirms only Converse/ConverseStream are traced; no invokeModel wrapping exists
  • braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/test/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustAWSBedrockTest.java — all tests use ConverseRequest and ConverseStreamRequest; no InvokeModelRequest test exists
  • braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.javatagBedrockRequest() and tagBedrockResponse() only handle Converse-shaped JSON (messages, output.message, usage.inputTokens); InvokeModel request/response bodies have entirely different schemas
Dominant language
Java
Stars
21
Forks
5
Avg merge
2d 7h
Merged PRs (30d)
8

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from braintrustdata/braintrust-sdk-java

All issues in braintrustdata/braintrust-sdk-java

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.