[bot] Anthropic Messages Batch API (`client.messages.batches`) not instrumented
还没有人认领这个 Issue。
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 48/100
- Issue 类型
- 功能
- 描述清晰度
- 基本清楚
- 活跃度
- 冷清
- 技术栈
- ruby
- 领域
- ai, observability
调研方向
从 lib/braintrust/contrib/anthropic/patcher.rb 和 lib/braintrust/contrib/anthropic/instrumentation/messages.rb 开始，然后比较现有的 create() 和 stream() instrumentation。跟踪 issue 中描述的 Anthropic Messages Batches create() 和 results() surface，包括 custom_id 关联。完成的标准是：batch 提交和每项结果都通过 spans 暴露所列出的输入、输出、token metrics 和 status，并且 coverage 与现有的 Messages integration 相匹配。
由索引模型根据 Issue 内容生成。
描述
Summary
The Anthropic Ruby SDK provides client.messages.batches for submitting multiple message requests for asynchronous Claude inference. This is a generative AI execution surface — each item in a batch runs the Claude model and produces a completion — but it is not instrumented. The SDK currently instruments the synchronous (messages.create) and streaming (messages.stream) paths for Anthropic, but not the batch path.
What is missing
The Messages Batch API (Anthropic::Resources::Messages::Batches) exposes these generative execution methods:
batches.create(requests: [...])
Submits up to 10,000 message requests for asynchronous processing. Each request item is a full MessagesCreateParams with a caller-provided custom_id. Returns a MessageBatch with a processing_status and a results_url.
batch = client.messages.batches.create(
requests: [
{ custom_id: "req-1", params: { model: "claude-opus-4-5", max_tokens: 1024, messages: [...] } },
{ custom_id: "req-2", params: { model: "claude-opus-4-5", max_tokens: 1024, messages: [...] } }
]
)
batches.results(batch_id)
Streams back the completed results once the batch finishes processing. Each result item contains a custom_id, a result type (succeeded, errored, expired), and for successes a full Message response object including content, usage tokens, model, and stop reason.
What instrumentation should capture
Because batch execution is asynchronous, tracing requires instrumenting both sides:
At submit time (batches.create):
- Each request item as input (messages, system prompt, model, max_tokens, tools, tool_choice, etc.)
- Batch-level metadata: total request count, model(s), custom IDs
At result time (batches.results):
- Each result item's output (content blocks, stop_reason)
- Per-item token metrics (input_tokens, output_tokens, cache tokens) from the Message usage object
- Per-item success/error status
A reasonable implementation is to emit one Braintrust span per batch result item when results are retrieved, carrying the original input from the corresponding custom_id request, matching the pattern the sync Messages integration already uses per-call.
Braintrust docs status
not_found — The Braintrust Anthropic integration docs at https://www.braintrust.dev/docs/integrations/ai-providers/anthropic and the Ruby SDK docs at https://www.braintrust.dev/docs do not mention batch API tracing. No Ruby-side batch instrumentation is documented anywhere in Braintrust's published docs.
Upstream sources
- Anthropic Messages Batch API docs: https://platform.claude.com/docs/en/api/messages-batches
- Anthropic Ruby SDK
Anthropic::Resources::Messages::Batches: https://github.com/anthropics/anthropic-sdk-ruby - Batch API reference in Anthropic SDK: available since anthropic gem v0.x stable, widely used for high-throughput eval and offline processing
Local repo files inspected
lib/braintrust/contrib/anthropic/patcher.rb— definesMessagesPatcherandBetaMessagesPatcher; noBatchesPatcherlib/braintrust/contrib/anthropic/instrumentation/messages.rb— wrapscreate()andstream()onAnthropic::Resources::Messages; nobatcheswrapperlib/braintrust/contrib/anthropic/instrumentation/beta_messages.rb— wrapscreate()andstream()on beta messages; no batches wrapper- Grep for
batches,MessageBatch,batch_id,custom_idacrosslib/braintrust/returns zero matches
- 主要语言
- Ruby
- 星标
- 9
- 派生
- 11
- 平均合并
- 2 天 3 小时
- 30 天内合并 PR
- 5
环境准备
- 提供 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
braintrustdata/braintrust-sdk-ruby 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 78/100
-
难度 2/5 1-3 小时 新手友好度 76/100
-
难度 2/5 1-3 小时 新手友好度 72/100
-
ruby
难度 3/5 半天 新手友好度 62/100
-
ruby
难度 3/5 1-2 天 新手友好度 45/100
查看 braintrustdata/braintrust-sdk-ruby 的全部 Issue
相似的 Issue
-
难度 1/5 1-3 小时 新手友好度 66/100
betagouv/conseillers-entreprises#4770 ·
维护者通常 1 天内回复
-
难度 1/5 1 小时以内 新手友好度 85/100
seek4science/seek#2762 ·
维护者通常 5 天内回复
-
难度 2/5 1-3 小时 新手友好度 67/100
Freika/dawarich#3910 · 3 个 reaction ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 75/100
rubyevents/rubyevents#2277 ·
维护者通常 1 天内回复
-
难度 1/5 1-3 小时 新手友好度 92/100
维护者通常 2 天内回复