[bot] OpenAI Batch API is not instrumented (mis-tagged as a generic LLM span)

Bắt đầu với braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java, đặc biệt là tagOpenAIRequest(), tagOpenAIResponse() và getSpanName(), sau đó kiểm tra BraintrustOpenAI.java và TracingHttpClient.java trong module OpenAI. Tìm trong các test instrumentation hiện có cách xử lý batch tương tự của provider; công việc hoàn tất khi các thao tác của Batch API và dữ liệu kết quả batch hiện có nhận được các span phù hợp, đồng thời có coverage cho create, retrieve, list và cancel.

Summary

The OpenAI instrumentation module (openai_2_15_0) generically intercepts every HTTP call via TracingHttpClient (swapped into ClientOptions.httpClient/originalHttpClient), so a call to client.batches().create(...) (or .retrieve()/.list()/.cancel()) does produce a span — but the shared tagging logic in InstrumentationSemConv has no awareness of the Batch API's request/response shape, so the span is actively mis-tagged rather than simply absent: it's marked span_attributes.type = "llm" as if it were a real model call, given a low-information span name ("batches"), and gets no model metadata, no input_json, and no output_json/metrics.

The OpenAI Batch API lets you submit up to 50,000 chat-completion/embeddings/responses/moderation requests as a single async job (POST /v1/batches); the job's eventual output file contains one JSONL line per request with the same generative output (token usage, model output) that this SDK already spans for synchronous calls — but none of that ever gets a Braintrust span today.

What is missing

In braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java:

• getSpanName() (lines 518–529) switches on providerName + ":" + lastPathSegment. For POST /v1/batches, the last path segment is "batches", which matches neither the openai:completions nor openai:embeddings case, so it falls through to default -> lastSegment, yielding the literal span name "batches" instead of something descriptive like "openai.batches.create".
• tagOpenAIRequest() (lines 110–141) unconditionally sets span_attributes = {"type":"llm"} (line 119) even though a batch-create call isn't itself a model invocation. It only reads metadata.model when requestJson.has("model") (line 129) and input_json from messages or an array-typed input (lines 133–137) — but a BatchCreateParams request body has none of these; it has input_file_id, endpoint (e.g. /v1/chat/completions), and completion_window. All of that is silently dropped.
• tagOpenAIResponse() (lines 143–208) looks for choices or output for output_json (lines 149–153) and a top-level usage object for metrics (line 160) — a Batch object (returned by create/retrieve/list) has neither; it has id, status, output_file_id, error_file_id, request_counts, and timestamps. None of this is captured, and there is no instrumentation at all of retrieving/parsing the completed batch's output file (where the actual per-request custom_id + generative response.body results, including usage, become available) — so even a fully successful, completed batch job produces zero spans reflecting its actual generative work.
• No test or example anywhere in the repo exercises client.batches() in any form (confirmed via grep for batch under braintrust-sdk/instrumentation/openai_2_15_0/ — zero matches).

Braintrust docs status: not_found

Checked https://www.braintrust.dev/docs/integrations/ai-providers/openai across all per-language sections (TypeScript, Python, Ruby, Go, Java, .NET): no mention of "batch" or "batches" anywhere. Its "What Braintrust traces" tables list only Chat Completion, Embedding, Moderation, openai.responses.create/parse/compact, Transcription, Translation, Speech, and Image Generation/Edit/Variation — the Batch API is absent for every language, not just Java. A broader site search only surfaces unrelated uses of "batch" (eval batches, UI batch labeling, batch-ingested span timestamps in the changelog).

Note: this repo's own gap-audit history already treats "Batch API not instrumented, mis-tagged as a generic LLM span" as a valid, in-scope finding — see the already-filed and still-open #155 for Anthropic's Message Batches API, which this issue mirrors for the OpenAI provider (a distinct upstream API/SDK, not a duplicate).

Upstream sources

• Official OpenAI Batch API reference: https://platform.openai.com/docs/api-reference/batch (create/retrieve/list/cancel sub-pages) and guide: https://platform.openai.com/docs/guides/batch — submit up to 50,000 requests (200MB input file) for async processing, typically within 24h, at a 50% cost discount vs. synchronous calls.
• Official openai-java SDK exposes this directly: openai-java-core/src/main/kotlin/com/openai/services/blocking/BatchService.kt and BatchServiceImpl.kt (https://github.com/openai/openai-java) define create(BatchCreateParams): Batch (POST /batches), retrieve(batchId), list(), and cancel(batchId) (POST /batches/{batch_id}/cancel). Also documented at https://developers.openai.com/api/reference/java/resources/batches.
• Request/response shape: create request has input_file_id, endpoint (/v1/chat/completions, /v1/embeddings, /v1/responses, or /v1/moderations), completion_window (currently only "24h"); output JSONL lines (fetched via output_file_id) have custom_id, response.body (the real chat-completion/embedding/responses result including usage), and error — structurally analogous to Anthropic's Message Batches results. Source: https://developers.openai.com/api/docs/guides/batch.

Local repo files inspected

• braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java — lines 110–141 (tagOpenAIRequest), 143–208 (tagOpenAIResponse), 518–529 (getSpanName)
• braintrust-sdk/instrumentation/openai_2_15_0/src/main/java/dev/braintrust/instrumentation/openai/v2_15_0/BraintrustOpenAI.java and TracingHttpClient.java — generic transport-swap; produces a span for any OpenAI HTTP call including /v1/batches, with no batch-specific logic
• Repo-wide grep for batch/Batch under braintrust-sdk/instrumentation/openai_2_15_0/ — zero matches (no test or example exercises this API)