[bot] Anthropic Message Batches API is not instrumented (mis-tagged as a generic LLM span)

Bắt đầu với InstrumentationSemConv.java, đặc biệt là tagAnthropicRequest(), tagAnthropicResponse() và getSpanName(), sau đó kiểm tra TracingHttpClient.java và ContextCapturingProxy.java để hiểu luồng span hiện có. Xem lại BraintrustAnthropicTest.java và các cấu trúc request cũng như kết quả batch của Anthropic trong tài liệu upstream được liên kết. Công việc được xem là hoàn tất khi các thao tác batch có span và metadata có ý nghĩa, đồng thời coverage của resultsStreaming() xác minh các kết quả thành công và lỗi cho từng request.

Summary

The Anthropic instrumentation module (anthropic_2_2_0) only has shape-aware tagging for the Messages API (messages().create/streaming). It has no support at all for the Message Batches API (client.messages().batches()), a stable, GA, actively-developed part of the Anthropic Java SDK for submitting many message-generation requests as a single asynchronous job.

Because the generic transport-swap (TracingHttpClient) still intercepts the batch HTTP calls, a batches().create(...) call does produce a span — but it is actively mis-tagged rather than simply absent: it gets span_attributes.type = "llm" (as if it were a real model call), a low-information span name ("batches", from the path-segment fallback), no model in metadata, and no input_json at all — because the request/response shapes for batches are structurally different from a single Messages call and none of the existing field checks match them.

What is missing

In braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java:

• getSpanName() (lines 511–522) switches on provider + ":" + lastPathSegment. For POST /v1/messages/batches, the last segment is "batches", which doesn't match the PROVIDER_NAME_ANTHROPIC + ":messages" case, so it falls through to default -> lastSegment, yielding the span name "batches" instead of something like "anthropic.messages.batches.create".
• tagAnthropicRequest() (lines 208–255) unconditionally sets span_attributes = {"type":"llm"} (line 216) even though a batch-create call isn't itself a model invocation. It only populates metadata.model and braintrust.input_json when requestJson.has("model") / requestJson.has("messages") (lines 231, 235) — but a BatchCreateParams request body has neither at the top level; it has a requests[] array where each entry is {"custom_id": "...", "params": {"model": ..., "messages": ..., "max_tokens": ..., ...}}. Every one of the (potentially many) inline generation requests in the batch is silently dropped from the span.
• tagAnthropicResponse() (lines 258–301) dumps the whole response body as output_json (harmless for batch-create, since the response is just batch job metadata: id, processing_status, request_counts, results_url) and only extracts metrics from a top-level usage object (line 270), which a batch-create/retrieve response never has — so no metrics are produced (correctly, since none exist yet at creation time, but there's also no instrumentation of the batch results retrieval, which is where the actual per-request usage and message outputs become available).
• There is no wrapping at all of batches().results(...)/resultsStreaming(...), so even after a batch completes, iterating its per-request results (each containing a custom_id and either a succeeded message with full usage, or an error) produces zero spans — unlike a normal messages().create() call, none of the individual generation results in a batch get any Braintrust span.
• No test or example anywhere in the repo exercises client.messages().batches() in any form (confirmed via repo-wide grep for batch/Batch in anthropic_2_2_0 — zero matches).

Braintrust docs status: supported (in a sibling SDK) / not_found (for Java)

• The Java section of https://www.braintrust.dev/docs/integrations/ai-providers/anthropic states only: "Braintrust emits spans for the Anthropic Messages API. Each span captures the input messages and response content," with a spans table listing exactly one row — "Anthropic Messages API spans" covering messages().create() calls including streaming. Batches are not mentioned for Java at all: not_found.
• The Python section of the same page states: "Braintrust emits spans for the Anthropic SDK's messages, batches, and managed agents APIs," with a spans-table row for anthropic.messages.batches.* explicitly covering the Batches API: supported (in the Python SDK). This establishes Message Batches tracing as an existing, documented Braintrust capability that the Java SDK has not brought to parity.

Upstream sources

• Anthropic Java SDK Batches sub-client: client.messages().batches().create(BatchCreateParams), .retrieve(...), .list(...), .cancel(...), .resultsStreaming(BatchResultsParams) — official Java code samples in the batch-processing guide: https://platform.claude.com/docs/en/build-with-claude/batch-processing (mirrored at https://docs.anthropic.com/en/docs/build-with-claude/batch-processing)
• Batch create request shape — each requests[] entry has custom_id + params containing "the standard Messages API parameters" (model, max_tokens, messages, system, tools, thinking, etc.), i.e. inline generation requests, not file/S3 references: same guide, "Prepare and create your batch" section
• Batch results shape — streamed JSONL keyed by custom_id, each with result.type of succeeded (containing the full message), errored, canceled, or expired: https://platform.claude.com/docs/en/api/messages/batches/results
• Currency/stability — Anthropic's official API release notes confirm this is an actively maintained, GA surface (e.g. March 30, 2026 entry raising max_tokens to 300k specifically "on the Message Batches API"; the Claude-on-AWS platform announcement listing "the full Messages API, Files API, Message Batches API, Claude Managed Agents" as core stable surfaces): https://platform.claude.com/docs/en/release-notes/api

Local files inspected

• braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java — lines 208–255 (tagAnthropicRequest: no top-level model/messages in a batch-create body, so metadata/input_json stay empty), lines 258–301 (tagAnthropicResponse: whole-body dump plus usage-only metrics, batch-create/retrieve responses have neither usage nor useful whole-body content), lines 511–522 (getSpanName: "batches" path segment falls through to the generic default case)
• braintrust-sdk/instrumentation/anthropic_2_2_0/src/main/java/dev/braintrust/instrumentation/anthropic/v2_2_0/TracingHttpClient.java and ContextCapturingProxy.java — generic transport-swap/service-graph-following wrapper; produces a span for any Anthropic HTTP call including batches, but has no batch-specific logic
• braintrust-sdk/instrumentation/anthropic_2_2_0/src/test/java/dev/braintrust/instrumentation/anthropic/v2_2_0/BraintrustAnthropicTest.java and BraintrustAnthropicPromptCachingTest.java — no test exercises batches() in any form
• examples/anthropic-instrumentation/ — no batch example exists
• Repo-wide grep for batch/Batch under braintrust-sdk/instrumentation/anthropic_2_2_0/ — zero matches