Hacktoberfest 2026:维护者为十月标记出来的 issue,仍然开放、适合新手。 浏览 Hacktoberfest issue

[bot] AWS Bedrock InvokeModel for embeddings not instrumented

未关闭
#134 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

评估

难度
4/5
预计耗时
3-5 天
新手友好度
68/100
Issue 类型
功能
描述清晰度
基本清楚
活跃度
冷清
技术栈
aws, java

调研方向

从 BraintrustBedrockInterceptor.java 和 BraintrustAWSBedrock.java 开始,跟踪 Converse 操作的选择方式,然后阅读 InstrumentationSemConv.java,了解现有的 Bedrock 请求和响应标记。检查 BraintrustAWSBedrockTest.java,并使用 Titan 和 Cohere 形式的 payload 为 InvokeModelRequest 添加覆盖测试。完成标准是 embedding 调用能够创建包含模型/提供商元数据、已捕获输入、embedding 类型以及指定 token 指标的 span。

由索引模型根据 Issue 内容生成。

描述

Summary

The AWS Bedrock instrumentation in BraintrustBedrockInterceptor explicitly skips all operations other than Converse and ConverseStream. This means embedding model calls — which use the InvokeModel API, not Converse — produce no spans, no input capture, and no token metrics.

All embedding models on AWS Bedrock require InvokeModel:

  • Amazon Titan Text Embeddings v2 (amazon.titan-embed-text-v2:0)
  • Amazon Titan Text Embeddings v1 (amazon.titan-embed-text-v1)
  • Amazon Titan Multimodal Embeddings (amazon.titan-embed-image-v1)
  • Cohere Embed English v3 (cohere.embed-english-v3)
  • Cohere Embed Multilingual v3 (cohere.embed-multilingual-v3)

None of these models support the Converse API — InvokeModel is the only invocation path for Bedrock embeddings.

What is missing

BraintrustBedrockInterceptor.beforeExecution() (lines 60–71) explicitly limits instrumentation to two operations:

private static final Set<String> INSTRUMENTED_OPERATIONS = Set.of("Converse", "ConverseStream");

// Only instrument Converse and ConverseStream — other Bedrock operations
// (InvokeModel, ApplyGuardrail, etc.) are not LLM calls we know how to tag.
if (!INSTRUMENTED_OPERATIONS.contains(operationName)) {
    return;
}

When a user calls client.invokeModel(...) with an embedding model (e.g. amazon.titan-embed-text-v2:0), the interceptor returns immediately without creating a span. No span is created, no input text is captured, and no embedding output or token metrics are recorded.

The BraintrustAWSBedrock.wrap() Javadoc on the sync builder explicitly states it traces "every converse call" and the async builder traces "every converseStream call" — embedding calls via InvokeModel are not in scope at all today.

A minimal InvokeModel embedding span should capture:

  • braintrust.metadata: model ID (from the request URI path /model/{modelId}/invoke), provider
  • braintrust.input_json: the inputText (or equivalent input field) from the request body
  • braintrust.metrics: prompt_tokens from the response inputTextTokenCount (Titan) or meta.inputTokenCount (Cohere)
  • braintrust.span_attributes: {"type": "embedding"}

Braintrust docs status

  • AWS Bedrock listed as a supported cloud provider at https://www.braintrust.dev/docs/integrations/ai-providers: supported (for generative use)
  • No mention of Bedrock embeddings or InvokeModel instrumentation in any Braintrust docs page: not_found
  • For comparison, OpenAI and Google GenAI embedding instrumentation exists in this repo (though with detail gaps tracked in #66 and #65)

Upstream sources

Local files inspected

  • braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustBedrockInterceptor.java — line 60: Set.of("Converse", "ConverseStream"); lines 63–71: beforeExecution returns without span for any other operation name including InvokeModel
  • braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustAWSBedrock.java — Javadoc on both wrap() overloads confirms only Converse/ConverseStream are traced; no invokeModel wrapping exists
  • braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/test/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustAWSBedrockTest.java — all tests use ConverseRequest and ConverseStreamRequest; no InvokeModelRequest test exists
  • braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java — tagBedrockRequest() and tagBedrockResponse() only handle Converse-shaped JSON (messages, output.message, usage.inputTokens); InvokeModel request/response bodies have entirely different schemas
主要语言
Java
星标
21
派生
5
平均合并
1 天 20 小时
30 天内合并 PR
8

环境准备

  • 没有 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

braintrustdata/braintrust-sdk-java 的其他 Issue

查看 braintrustdata/braintrust-sdk-java 的全部 Issue

相似的 Issue

更多 Java Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。