[bot] AWS Bedrock InvokeModel for embeddings not instrumented
还没有人认领这个 Issue。
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 68/100
- Issue 类型
- 功能
- 描述清晰度
- 基本清楚
- 活跃度
- 冷清
- 技术栈
- aws, java
- 领域
- cloud, observability
调研方向
从 BraintrustBedrockInterceptor.java 和 BraintrustAWSBedrock.java 开始,跟踪 Converse 操作的选择方式,然后阅读 InstrumentationSemConv.java,了解现有的 Bedrock 请求和响应标记。检查 BraintrustAWSBedrockTest.java,并使用 Titan 和 Cohere 形式的 payload 为 InvokeModelRequest 添加覆盖测试。完成标准是 embedding 调用能够创建包含模型/提供商元数据、已捕获输入、embedding 类型以及指定 token 指标的 span。
由索引模型根据 Issue 内容生成。
描述
Summary
The AWS Bedrock instrumentation in BraintrustBedrockInterceptor explicitly skips all operations other than Converse and ConverseStream. This means embedding model calls — which use the InvokeModel API, not Converse — produce no spans, no input capture, and no token metrics.
All embedding models on AWS Bedrock require InvokeModel:
- Amazon Titan Text Embeddings v2 (
amazon.titan-embed-text-v2:0) - Amazon Titan Text Embeddings v1 (
amazon.titan-embed-text-v1) - Amazon Titan Multimodal Embeddings (
amazon.titan-embed-image-v1) - Cohere Embed English v3 (
cohere.embed-english-v3) - Cohere Embed Multilingual v3 (
cohere.embed-multilingual-v3)
None of these models support the Converse API — InvokeModel is the only invocation path for Bedrock embeddings.
What is missing
BraintrustBedrockInterceptor.beforeExecution() (lines 60–71) explicitly limits instrumentation to two operations:
private static final Set<String> INSTRUMENTED_OPERATIONS = Set.of("Converse", "ConverseStream");
// Only instrument Converse and ConverseStream — other Bedrock operations
// (InvokeModel, ApplyGuardrail, etc.) are not LLM calls we know how to tag.
if (!INSTRUMENTED_OPERATIONS.contains(operationName)) {
return;
}
When a user calls client.invokeModel(...) with an embedding model (e.g. amazon.titan-embed-text-v2:0), the interceptor returns immediately without creating a span. No span is created, no input text is captured, and no embedding output or token metrics are recorded.
The BraintrustAWSBedrock.wrap() Javadoc on the sync builder explicitly states it traces "every converse call" and the async builder traces "every converseStream call" — embedding calls via InvokeModel are not in scope at all today.
A minimal InvokeModel embedding span should capture:
braintrust.metadata: model ID (from the request URI path/model/{modelId}/invoke), providerbraintrust.input_json: theinputText(or equivalent input field) from the request bodybraintrust.metrics:prompt_tokensfrom the responseinputTextTokenCount(Titan) ormeta.inputTokenCount(Cohere)braintrust.span_attributes:{"type": "embedding"}
Braintrust docs status
- AWS Bedrock listed as a supported cloud provider at https://www.braintrust.dev/docs/integrations/ai-providers: supported (for generative use)
- No mention of Bedrock embeddings or
InvokeModelinstrumentation in any Braintrust docs page: not_found - For comparison, OpenAI and Google GenAI embedding instrumentation exists in this repo (though with detail gaps tracked in #66 and #65)
Upstream sources
- AWS Bedrock InvokeModel API reference: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html — "You use model inference to generate text, images, and embeddings."
- Amazon Titan Text Embeddings v2 example: https://docs.aws.amazon.com/bedrock/latest/userguide/bedrock-runtime_example_bedrock-runtime_InvokeModel_TitanTextEmbeddings_section.html — uses
client.invokeModel()with model IDamazon.titan-embed-text-v2:0; response includesembeddingarray andinputTextTokenCount - Cohere Embed on Bedrock: https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-embed.html — uses
InvokeModelwithtextsarray; response includesembeddingsandmeta.inputTokenCount - AWS SDK for Java v2 BedrockRuntimeClient:
invokeModel(InvokeModelRequest)— operation nameInvokeModel, same SDK client as Converse
Local files inspected
braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustBedrockInterceptor.java— line 60:Set.of("Converse", "ConverseStream"); lines 63–71:beforeExecutionreturns without span for any other operation name includingInvokeModelbraintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustAWSBedrock.java— Javadoc on bothwrap()overloads confirms only Converse/ConverseStream are traced; noinvokeModelwrapping existsbraintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/test/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustAWSBedrockTest.java— all tests useConverseRequestandConverseStreamRequest; noInvokeModelRequesttest existsbraintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java—tagBedrockRequest()andtagBedrockResponse()only handle Converse-shaped JSON (messages, output.message, usage.inputTokens); InvokeModel request/response bodies have entirely different schemas
- 主要语言
- Java
- 星标
- 21
- 派生
- 5
- 平均合并
- 1 天 20 小时
- 30 天内合并 PR
- 8
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
braintrustdata/braintrust-sdk-java 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 82/100
-
难度 2/5 1-3 小时 新手友好度 82/100
-
难度 2/5 1-3 小时 新手友好度 82/100
-
难度 2/5 1-3 小时 新手友好度 78/100
-
难度 2/5 1-3 小时 新手友好度 68/100
查看 braintrustdata/braintrust-sdk-java 的全部 Issue
相似的 Issue
-
bug
难度 2/5 1-3 小时 新手友好度 76/100
lugassawan/swe-workbench#799 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 66/100
Vakore/ZappierGames#89 ·
-
1.20.1-FORGE Module-Enhancement
难度 2/5 1-3 小时 新手友好度 65/100
MehVahdJukaar/WoodGood#1340 · 1 条评论 ·
-
enhancement
难度 2/5 1-3 小时 新手友好度 65/100
PssbleTrngle/dye_depot#123 ·
-
bug question
难度 2/5 1-3 小时 新手友好度 67/100