[bot] Spring AI tool/function execution produces no span — only the surrounding chat model call is traced
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 38/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- java
- Domain
- ai, observability
Research direction
Start with BraintrustSpringAI.java and the corresponding auto/*InstrumentationModule.java files in both springai_1_0_0 and springai_2_0_0, then compare langchain_1_8_0's TracingToolExecutor. Trace Spring AI's ToolCallingManager and ToolCallback execution path. Done means each local tool invocation has a dedicated span in both supported modules, alongside the existing model-call spans.
Written by the indexing model from the issue text.
Description
Summary
Both Spring AI instrumentation modules (springai_1_0_0, springai_2_0_0) only wrap OpenAiChatModel.Builder/AnthropicChatModel.Builder construction (BraintrustSpringAI.wrap()). Spring AI's own local tool-execution machinery — ToolCallingManager/DefaultToolCallingManager, the auto-registered ToolCallingAdvisor, and ToolCallback/MethodToolCallback/FunctionToolCallback — is never touched by either module. When a model response triggers a tool call, Spring AI resolves and invokes the corresponding Java method in-process, but that invocation produces no span: only the LLM call that requested the tool and the follow-up LLM call that consumes the result are traced, with the actual tool execution invisible in between.
This is distinct from the already-filed streaming-reassembly bugs (#78 Anthropic SSE tool_use/thinking blocks, #79 OpenAI SSE tool_calls delta-merging) — those are about the LLM span's own tool_calls output being malformed during streaming. This gap is about the absence of a separate span for the tool's actual execution, which is the "tool-calling during model execution" surface itself, comparable to langchain_1_8_0's TracingToolExecutor which does instrument LangChain4j's equivalent local tool execution.
What is missing
A full-repo search for ToolCalling, ToolCallback, and toolCall under both springai_1_0_0 and springai_2_0_0 returns zero matches. BraintrustSpringAI.java in both modules exposes exactly one entry point:
public static <T> T wrap(OpenTelemetry openTelemetry, T chatModelBuilder) { ... }
which only instruments chat model construction. Neither module registers any instrumentation for:
ToolCallingManager.executeToolCalls()— the method that actually resolves and invokes tool callbacks in response to a model's tool-call requestToolCallingAdvisor— theChatClientadvisor auto-registered whenever tools are present, which drives the resolve/execute loopToolCallback.call()(and itsMethodToolCallback/FunctionToolCallbackimplementations) — the actual per-tool invocation point
For comparison, langchain_1_8_0 instruments LangChain4j's analogous local-execution surface via TracingToolExecutor, producing a dedicated span per tool call — Spring AI has no equivalent.
Braintrust docs status
- Spring AI integration is not documented on Braintrust's integrations page at https://www.braintrust.dev/docs/integrations/ai-providers: not_found
- Braintrust's general tracing docs (https://www.braintrust.dev/docs/instrument/trace-llm-calls) describe function/tool calling as part of LLM span capture, not as a separately-spanned execution step: not_found for a documented tool-execution span pattern this gap would need to follow
Upstream sources
- Spring AI tool calling reference: https://docs.spring.io/spring-ai/reference/api/tools.html — documents
ToolCallingManager,ToolCallingAdvisor(auto-registered in theChatClientadvisor chain), andToolCallback/MethodToolCallback/FunctionToolCallbackas the current (Spring AI 1.x/2.x) tool-execution architecture - Spring AI OpenAI chat reference: https://docs.spring.io/spring-ai/reference/api/chat/openai-chat.html — confirms function/tool calling support for the exact provider this repo already instruments
Local files inspected
braintrust-sdk/instrumentation/springai_1_0_0/src/main/java/dev/braintrust/instrumentation/springai/v1_0_0/BraintrustSpringAI.java— singlewrap(OpenTelemetry, T chatModelBuilder)entry point (line 61); no tool-calling instrumentationbraintrust-sdk/instrumentation/springai_1_0_0/src/main/java/dev/braintrust/instrumentation/springai/v1_0_0/auto/SpringAIOpenAIInstrumentationModule.javaandSpringAIAnthropicInstrumentationModule.java— only instrument the two chat model buildersbraintrust-sdk/instrumentation/springai_2_0_0/src/main/java/dev/braintrust/instrumentation/springai/v2_0_0/BraintrustSpringAI.java,SpringAIOpenAI.java,SpringAIAnthropic.java, and theirauto/*InstrumentationModule.java— same scope as 1.0.0, no tool-calling instrumentation- Full-repo grep for
ToolCalling,ToolCallback,toolCallscoped to both Spring AI modules — zero matches braintrust-sdk/instrumentation/langchain_1_8_0/— confirmed as the comparison point: this module does instrument LangChain4j's equivalent local tool-execution surface (TracingToolExecutor)
- Dominant language
- Java
- Stars
- 21
- Forks
- 5
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 8
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from braintrustdata/braintrust-sdk-java
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
All issues in braintrustdata/braintrust-sdk-java
Similar issues
-
awaiting triage bug Causes friction Hop Gui P1 P2 Transforms
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
apache/flink-agents#1152 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
jenkinsci/blueocean-plugin#5417 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
objectionary/eo-graphs#75 ·