[bot] Bedrock Converse prompt caching metrics (cacheReadInputTokens, cacheWriteInputTokens) not captured
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 3/5
- Tiempo estimado
- 1-2 días
- Aptitud para principiantes
- 70/100
- Tipo de issue
- Error
- Claridad
- Bien especificado
- Estado de actividad
- Tranquilo
- Stack tecnológico
- aws, java
- Área
- observability
Línea de trabajo
Comienza en InstrumentationSemConv.java, en tagBedrockResponse(), y luego sigue parseTokenUsage() y buildConverseJson() en BraintrustBedrockInterceptor.java para la ruta de streaming. Añade cobertura en BraintrustAWSBedrockTest.java usando una respuesta con campos de uso de la caché de prompts de Bedrock; la tarea está terminada cuando ambas rutas de Converse conservan y capturan cacheReadInputTokens, cacheWriteInputTokens y cacheDetails.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Summary
The Bedrock Converse instrumentation extracts inputTokens, outputTokens, and totalTokens from the response usage object, but silently drops the prompt caching fields cacheReadInputTokens, cacheWriteInputTokens, and cacheDetails. These fields are returned by the Bedrock Converse API when prompt caching is active and are important for understanding cache hit rates and cost savings.
Both the non-streaming (Converse) and streaming (ConverseStream) paths are affected.
What is missing
Non-streaming path
In InstrumentationSemConv.tagBedrockResponse() (lines 350–357), only three usage fields are extracted:
if (usage.has("inputTokens")) metrics.put("prompt_tokens", usage.get("inputTokens"));
if (usage.has("outputTokens")) metrics.put("completion_tokens", usage.get("outputTokens"));
if (usage.has("totalTokens")) metrics.put("tokens", usage.get("totalTokens"));
The following fields from the Bedrock usage object are never extracted:
cacheReadInputTokens— tokens served from the prompt cachecacheWriteInputTokens— tokens written to the prompt cachecacheDetails— array of per-checkpoint cache details including TTL
Streaming path
In BraintrustBedrockInterceptor.TeeingSubscriber.parseTokenUsage() (lines 362–379), only inputTokens and outputTokens are parsed from the metadata event payload. Cache token fields in the same payload are ignored. The buildConverseJson() method (lines 385–410) then constructs a synthetic response with only inputTokens, outputTokens, and totalTokens — cache fields are lost before they reach tagBedrockResponse.
A real Converse response with prompt caching looks like:
"usage": {
"inputTokens": 1200,
"outputTokens": 350,
"totalTokens": 1550,
"cacheReadInputTokens": 800,
"cacheWriteInputTokens": 400,
"cacheDetails": [
{ "inputTokens": 800, "ttl": "5m" }
]
}
Today, only inputTokens, outputTokens, and totalTokens are captured. The cache fields are silently dropped.
For comparison, the Google GenAI handler in this repo already extracts cachedContentTokenCount as prompt_cached_tokens (line 142–146 of BraintrustApiClient.java), showing that cache token extraction is an established pattern here. Similar gaps for Anthropic (#57) and OpenAI (#58, #70) cache tokens have already been filed.
Braintrust docs status
- Braintrust lists AWS Bedrock as a supported cloud provider at https://www.braintrust.dev/docs/integrations/ai-providers
- The Bedrock-specific docs page does not mention prompt caching or cache token metrics: not_found
Upstream sources
- AWS Bedrock prompt caching docs: https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html — GA feature, documents
cacheReadInputTokensandcacheWriteInputTokensin Converse response usage - Bedrock Converse API reference: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html —
TokenUsageincludescacheReadInputTokens,cacheWriteInputTokens,cacheDetails - Supported models: Claude Opus 4, Claude 3.7 Sonnet (GA); Claude 3.5 Sonnet v2 (Preview); Amazon Nova (automatic caching)
Local files inspected
braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java— lines 350–357 (tagBedrockResponse: onlyinputTokens,outputTokens,totalTokensextracted from usage)braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustBedrockInterceptor.java— lines 362–379 (parseTokenUsage: onlyinputTokensandoutputTokensparsed); lines 385–410 (buildConverseJson: synthetic response omits cache fields)braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/test/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustAWSBedrockTest.java— no test exercises prompt caching responsesbraintrust-sdk/instrumentation/genai_1_18_0/src/main/java/com/google/genai/BraintrustApiClient.java— lines 142–146 (GenAI handler already extractscachedContentTokenCountasprompt_cached_tokens)
- Lenguaje dominante
- Java
- Estrellas
- 21
- Forks
- 5
- Merge medio
- 2 d 7 h
- PR fusionados (30 d)
- 8
Guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de braintrustdata/braintrust-sdk-java
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
Todos los issues de braintrustdata/braintrust-sdk-java
Issues similares
-
certification
Dificultad 1/5 Menos de una hora Aptitud para principiantes 80/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 75/100
-
[BUG] ECR GetAuthorizationToken returns a proxyEndpoint for the default region, not the request's Abiertobug ecr
Dificultad 2/5 1-3 horas Aptitud para principiantes 75/100
-
Needs: Triage Type: Feature request
Dificultad 2/5 1-3 horas Aptitud para principiantes 70/100
AntennaPod/AntennaPod#8794 ·
-
agentic-workflows
Dificultad 2/5 1-3 horas Aptitud para principiantes 65/100
github/copilot-sdk#2760 ·