[bot] Bedrock Converse prompt caching metrics (cacheReadInputTokens, cacheWriteInputTokens) not captured
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 70/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Quiet
- Tech stack
- aws, java
- Domain
- observability
Research direction
Start in InstrumentationSemConv.java at tagBedrockResponse(), then trace parseTokenUsage() and buildConverseJson() in BraintrustBedrockInterceptor.java for the streaming path. Add coverage in BraintrustAWSBedrockTest.java using a response with Bedrock prompt-cache usage fields; done means both Converse paths retain and capture cacheReadInputTokens, cacheWriteInputTokens, and cacheDetails.
Written by the indexing model from the issue text.
Description
Summary
The Bedrock Converse instrumentation extracts inputTokens, outputTokens, and totalTokens from the response usage object, but silently drops the prompt caching fields cacheReadInputTokens, cacheWriteInputTokens, and cacheDetails. These fields are returned by the Bedrock Converse API when prompt caching is active and are important for understanding cache hit rates and cost savings.
Both the non-streaming (Converse) and streaming (ConverseStream) paths are affected.
What is missing
Non-streaming path
In InstrumentationSemConv.tagBedrockResponse() (lines 350–357), only three usage fields are extracted:
if (usage.has("inputTokens")) metrics.put("prompt_tokens", usage.get("inputTokens"));
if (usage.has("outputTokens")) metrics.put("completion_tokens", usage.get("outputTokens"));
if (usage.has("totalTokens")) metrics.put("tokens", usage.get("totalTokens"));
The following fields from the Bedrock usage object are never extracted:
cacheReadInputTokens— tokens served from the prompt cachecacheWriteInputTokens— tokens written to the prompt cachecacheDetails— array of per-checkpoint cache details including TTL
Streaming path
In BraintrustBedrockInterceptor.TeeingSubscriber.parseTokenUsage() (lines 362–379), only inputTokens and outputTokens are parsed from the metadata event payload. Cache token fields in the same payload are ignored. The buildConverseJson() method (lines 385–410) then constructs a synthetic response with only inputTokens, outputTokens, and totalTokens — cache fields are lost before they reach tagBedrockResponse.
A real Converse response with prompt caching looks like:
"usage": {
"inputTokens": 1200,
"outputTokens": 350,
"totalTokens": 1550,
"cacheReadInputTokens": 800,
"cacheWriteInputTokens": 400,
"cacheDetails": [
{ "inputTokens": 800, "ttl": "5m" }
]
}
Today, only inputTokens, outputTokens, and totalTokens are captured. The cache fields are silently dropped.
For comparison, the Google GenAI handler in this repo already extracts cachedContentTokenCount as prompt_cached_tokens (line 142–146 of BraintrustApiClient.java), showing that cache token extraction is an established pattern here. Similar gaps for Anthropic (#57) and OpenAI (#58, #70) cache tokens have already been filed.
Braintrust docs status
- Braintrust lists AWS Bedrock as a supported cloud provider at https://www.braintrust.dev/docs/integrations/ai-providers
- The Bedrock-specific docs page does not mention prompt caching or cache token metrics: not_found
Upstream sources
- AWS Bedrock prompt caching docs: https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html — GA feature, documents
cacheReadInputTokensandcacheWriteInputTokensin Converse response usage - Bedrock Converse API reference: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html —
TokenUsageincludescacheReadInputTokens,cacheWriteInputTokens,cacheDetails - Supported models: Claude Opus 4, Claude 3.7 Sonnet (GA); Claude 3.5 Sonnet v2 (Preview); Amazon Nova (automatic caching)
Local files inspected
braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java— lines 350–357 (tagBedrockResponse: onlyinputTokens,outputTokens,totalTokensextracted from usage)braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustBedrockInterceptor.java— lines 362–379 (parseTokenUsage: onlyinputTokensandoutputTokensparsed); lines 385–410 (buildConverseJson: synthetic response omits cache fields)braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/test/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustAWSBedrockTest.java— no test exercises prompt caching responsesbraintrust-sdk/instrumentation/genai_1_18_0/src/main/java/com/google/genai/BraintrustApiClient.java— lines 142–146 (GenAI handler already extractscachedContentTokenCountasprompt_cached_tokens)
- Dominant language
- Java
- Stars
- 21
- Forks
- 5
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 8
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from braintrustdata/braintrust-sdk-java
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
All issues in braintrustdata/braintrust-sdk-java
Similar issues
-
awaiting triage bug Causes friction Hop Gui P1 P2 Transforms
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
apache/flink-agents#1152 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
jenkinsci/blueocean-plugin#5417 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
objectionary/eo-graphs#75 ·