Hacktoberfest 2026：维护者为十月标记出来的 issue，仍然开放、适合新手。 浏览 Hacktoberfest issue

[BOT ISSUE] Anthropic server_tool_use usage metrics silently dropped

未关闭 适合新手
  • 哪里不对？
#137 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
2/5
预计耗时
1-3 小时
新手友好度
72/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
冷清
技术栈
ruby
领域
observability

调研方向

从 lib/braintrust/contrib/anthropic/instrumentation/common.rb 中的 parse_usage_tokens 开始，并比较 messages.rb 和 beta_messages.rb 如何将其用于流式和非流式响应。确保 server_tool_use.web_search_requests 的使用量值保留在 Braintrust 指标中，包括通过两条响应路径。

由索引模型根据 Issue 内容生成。

描述

Summary

When Anthropic's built-in web search tool is used, the API returns a server_tool_use field in the usage object containing web_search_requests counts. The SDK's Common.parse_usage_tokens silently drops this field because it only maps 4 hardcoded usage fields. Users paying for web search ($10/1000 searches) have no visibility into their server-side tool usage through Braintrust metrics.

What is missing

The Anthropic Messages API returns usage like this when web search is active:

"usage": {
  "input_tokens": 105,
  "output_tokens": 6039,
  "cache_read_input_tokens": 7123,
  "cache_creation_input_tokens": 7345,
  "server_tool_use": {
    "web_search_requests": 1
  }
}

Common.parse_usage_tokens (lines 22–27) only maps these fields:

Anthropic field Braintrust metric
input_tokens prompt_tokens
output_tokens completion_tokens
cache_read_input_tokens prompt_cached_tokens
cache_creation_input_tokens prompt_cache_creation_tokens

The server_tool_use object is silently ignored. Any future server-side tool usage fields added by Anthropic will also be silently dropped.

Impact
  • Web search is a stable, GA Anthropic feature with two tool versions (web_search_20250305, web_search_20260209)
  • Each web search is billed at $10/1000 requests — a meaningful cost that users would want to track
  • The server_tool_use content block type already appears in response content arrays (captured in braintrust.output_json), but the aggregate usage count in the usage object is lost

Braintrust docs status

not_found — The Braintrust Anthropic integration docs at https://www.braintrust.dev/docs/integrations/ai-providers/anthropic document metric collection including cached tokens, but do not mention server_tool_use or web search request tracking.

Upstream sources

Local repo files inspected

  • lib/braintrust/contrib/anthropic/instrumentation/common.rb — parse_usage_tokens method (lines 14–48) defines a 4-field field_map and skips all unrecognized keys via next unless ... target
  • lib/braintrust/contrib/anthropic/instrumentation/messages.rb — calls Common.parse_usage_tokens for both streaming and non-streaming responses
  • lib/braintrust/contrib/anthropic/instrumentation/beta_messages.rb — same pattern
主要语言
Ruby
星标
9
派生
11
平均合并
3 天 43 分钟
30 天内合并 PR
9

环境准备

  • 提供 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

braintrustdata/braintrust-sdk-ruby 的其他 Issue

查看 braintrustdata/braintrust-sdk-ruby 的全部 Issue

相似的 Issue

更多 Ruby Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。