Hacktoberfest 2026：维护者为十月标记出来的 issue，仍然开放、适合新手。 浏览 Hacktoberfest issue

[bot] Anthropic tool_runner agentic loop not instrumented

未关闭
  • 哪里不对？
#153 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
4/5
预计耗时
3-5 天
新手友好度
58/100
Issue 类型
功能
描述清晰度
描述清楚
活跃度
冷清
技术栈
ruby
领域
observability

调研方向

先从 lib/braintrust/contrib/anthropic/patcher.rb 和 instrumentation/beta_messages.rb 开始，然后检查 issue 中提到的 Anthropic SDK 的 tool_runner 方法。对比 lib/braintrust/contrib/ruby_llm/instrumentation/chat.rb 中嵌套的 tool span 模式。完成的标准是 runner 具有一个整体 trace，每个 BaseTool#call 都有一个子 tool span，并且所有列出的执行模式都得到覆盖。

由索引模型根据 Issue 内容生成。

描述

Summary

The Anthropic Ruby SDK provides client.beta.messages.tool_runner(...), a beta agentic loop API that automatically executes tools and manages the multi-turn conversation cycle. This surface is not instrumented. The SDK currently instruments beta.messages.create() and beta.messages.stream() (via BetaMessagesPatcher), so individual LLM calls within the runner may produce spans, but the tool executions and overall agentic run are invisible in traces.

What is missing

The tool_runner API (Anthropic::Resources::Beta::Messages#tool_runner) returns a runner object with these execution methods:

  • each_message { |msg| ... } — iterates through the agentic loop, auto-executing tools via BaseTool#call between iterations
  • run_until_finished — runs the full loop and returns all messages
  • next_message — step-by-step manual iteration
  • each_streaming { |event| ... } — streaming variant of the agentic loop

At each iteration, the runner:

  1. Sends a message to Claude (this call IS traced via existing BetaMessagesPatcher)
  2. Detects tool_use blocks in Claude's response
  3. Executes BaseTool#call on the matching tool (NOT traced)
  4. Sends tool results back and loops
What instrumentation should capture

Parent span for the agentic run (e.g., anthropic.tool_runner):

  • Input: initial messages and tool definitions
  • Output: final response after all tool loops complete
  • Metrics: aggregate token usage across all iterations, total duration

Child spans for each tool execution (e.g., anthropic.tool.{tool_name}):

  • Input: tool name + arguments (from Claude's tool_use block)
  • Output: tool result (from BaseTool#call return value)
  • Span attributes: {type: "tool"}

This pattern is already established in this repo — the RubyLLM integration creates ruby_llm.tool.{tool_name} child spans with braintrust.span_attributes: {type: "tool"} for each tool execution in lib/braintrust/contrib/ruby_llm/instrumentation/chat.rb.

Braintrust docs status

  • Braintrust documents "tool" as a first-class span type: "A tool call made by the model — an external API, code execution, database query, etc." (tracing guide)
  • Ruby-specific tool_runner instrumentation: not_found

Upstream sources

Local repo files inspected

  • lib/braintrust/contrib/anthropic/patcher.rb — defines MessagesPatcher and BetaMessagesPatcher; no tool_runner patcher
  • lib/braintrust/contrib/anthropic/instrumentation/beta_messages.rb — wraps create() and stream() only; no tool_runner wrapper
  • Grep for tool_runner, BaseTool, each_message, run_until_finished across lib/ — zero matches
  • lib/braintrust/contrib/ruby_llm/instrumentation/chat.rb — demonstrates the existing pattern for tool execution tracing with nested spans
主要语言
Ruby
星标
9
派生
11
平均合并
3 天 43 分钟
30 天内合并 PR
9

环境准备

  • 提供 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

braintrustdata/braintrust-sdk-ruby 的其他 Issue

查看 braintrustdata/braintrust-sdk-ruby 的全部 Issue

相似的 Issue

更多 Ruby Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。