[bot] AWS Bedrock InvokeModel for embeddings not instrumented
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Aptitud para principiantes
- 68/100
- Tipo de issue
- Nueva funcionalidad
- Claridad
- Bastante claro
- Estado de actividad
- Tranquilo
- Stack tecnológico
- aws, java
- Área
- cloud, observability
Línea de trabajo
Empieza con BraintrustBedrockInterceptor.java y BraintrustAWSBedrock.java para rastrear cómo se seleccionan las operaciones de Converse y, después, lee InstrumentationSemConv.java para revisar el etiquetado existente de las solicitudes y respuestas de Bedrock. Revisa BraintrustAWSBedrockTest.java y añade cobertura para InvokeModelRequest usando payloads con la estructura de Titan y Cohere. La tarea estará completa cuando las llamadas de embedding creen spans con metadatos del modelo/proveedor, la entrada capturada, el tipo de embedding y las métricas de tokens especificadas.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Summary
The AWS Bedrock instrumentation in BraintrustBedrockInterceptor explicitly skips all operations other than Converse and ConverseStream. This means embedding model calls — which use the InvokeModel API, not Converse — produce no spans, no input capture, and no token metrics.
All embedding models on AWS Bedrock require InvokeModel:
- Amazon Titan Text Embeddings v2 (
amazon.titan-embed-text-v2:0) - Amazon Titan Text Embeddings v1 (
amazon.titan-embed-text-v1) - Amazon Titan Multimodal Embeddings (
amazon.titan-embed-image-v1) - Cohere Embed English v3 (
cohere.embed-english-v3) - Cohere Embed Multilingual v3 (
cohere.embed-multilingual-v3)
None of these models support the Converse API — InvokeModel is the only invocation path for Bedrock embeddings.
What is missing
BraintrustBedrockInterceptor.beforeExecution() (lines 60–71) explicitly limits instrumentation to two operations:
private static final Set<String> INSTRUMENTED_OPERATIONS = Set.of("Converse", "ConverseStream");
// Only instrument Converse and ConverseStream — other Bedrock operations
// (InvokeModel, ApplyGuardrail, etc.) are not LLM calls we know how to tag.
if (!INSTRUMENTED_OPERATIONS.contains(operationName)) {
return;
}
When a user calls client.invokeModel(...) with an embedding model (e.g. amazon.titan-embed-text-v2:0), the interceptor returns immediately without creating a span. No span is created, no input text is captured, and no embedding output or token metrics are recorded.
The BraintrustAWSBedrock.wrap() Javadoc on the sync builder explicitly states it traces "every converse call" and the async builder traces "every converseStream call" — embedding calls via InvokeModel are not in scope at all today.
A minimal InvokeModel embedding span should capture:
braintrust.metadata: model ID (from the request URI path/model/{modelId}/invoke), providerbraintrust.input_json: theinputText(or equivalent input field) from the request bodybraintrust.metrics:prompt_tokensfrom the responseinputTextTokenCount(Titan) ormeta.inputTokenCount(Cohere)braintrust.span_attributes:{"type": "embedding"}
Braintrust docs status
- AWS Bedrock listed as a supported cloud provider at https://www.braintrust.dev/docs/integrations/ai-providers: supported (for generative use)
- No mention of Bedrock embeddings or
InvokeModelinstrumentation in any Braintrust docs page: not_found - For comparison, OpenAI and Google GenAI embedding instrumentation exists in this repo (though with detail gaps tracked in #66 and #65)
Upstream sources
- AWS Bedrock InvokeModel API reference: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html — "You use model inference to generate text, images, and embeddings."
- Amazon Titan Text Embeddings v2 example: https://docs.aws.amazon.com/bedrock/latest/userguide/bedrock-runtime_example_bedrock-runtime_InvokeModel_TitanTextEmbeddings_section.html — uses
client.invokeModel()with model IDamazon.titan-embed-text-v2:0; response includesembeddingarray andinputTextTokenCount - Cohere Embed on Bedrock: https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-embed.html — uses
InvokeModelwithtextsarray; response includesembeddingsandmeta.inputTokenCount - AWS SDK for Java v2 BedrockRuntimeClient:
invokeModel(InvokeModelRequest)— operation nameInvokeModel, same SDK client as Converse
Local files inspected
braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustBedrockInterceptor.java— line 60:Set.of("Converse", "ConverseStream"); lines 63–71:beforeExecutionreturns without span for any other operation name includingInvokeModelbraintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustAWSBedrock.java— Javadoc on bothwrap()overloads confirms only Converse/ConverseStream are traced; noinvokeModelwrapping existsbraintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/test/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustAWSBedrockTest.java— all tests useConverseRequestandConverseStreamRequest; noInvokeModelRequesttest existsbraintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java—tagBedrockRequest()andtagBedrockResponse()only handle Converse-shaped JSON (messages, output.message, usage.inputTokens); InvokeModel request/response bodies have entirely different schemas
- Lenguaje dominante
- Java
- Estrellas
- 21
- Forks
- 5
- Merge medio
- 2 d 7 h
- PR fusionados (30 d)
- 8
Guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de braintrustdata/braintrust-sdk-java
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
Todos los issues de braintrustdata/braintrust-sdk-java
Issues similares
-
certification
Dificultad 1/5 Menos de una hora Aptitud para principiantes 80/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 75/100
-
[BUG] ECR GetAuthorizationToken returns a proxyEndpoint for the default region, not the request's Abiertobug ecr
Dificultad 2/5 1-3 horas Aptitud para principiantes 75/100
-
Needs: Triage Type: Feature request
Dificultad 2/5 1-3 horas Aptitud para principiantes 70/100
AntennaPod/AntennaPod#8794 ·
-
agentic-workflows
Dificultad 2/5 1-3 horas Aptitud para principiantes 65/100
github/copilot-sdk#2760 ·