[BOT ISSUE] Anthropic system prompt silently dropped from span input when using array format (prompt caching)

Open Beginner friendly
#81 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
82/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Quiet
Tech stack
java
Domain
observability

Research direction

Start in braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java at tagAnthropicRequest(), then inspect BraintrustAnthropicTest.java for existing request-tagging coverage. Verify that string and array-format system prompts are retained in braintrust.input_json, including the array format used with cache_control, for both shared streaming and non-streaming paths.

Written by the indexing model from the issue text.

Description

Summary

The Anthropic request tagger in InstrumentationSemConv.tagAnthropicRequest() silently drops the system prompt from braintrust.input_json when it is provided in array-of-content-blocks format. This array format is required when using Anthropic prompt caching on system prompts (each block needs a cache_control marker), and is explicitly documented by both Anthropic and Braintrust.

When system is a plain string (e.g. "system": "You are helpful"), it is correctly appended to the input array as a synthetic {role: "system", content: "..."} entry. But when system is an array (e.g. "system": [{"type":"text","text":"You are helpful","cache_control":{"type":"ephemeral"}}]), the guard condition fails and the system prompt is silently omitted.

What is missing

In InstrumentationSemConv.tagAnthropicRequest() (lines 192–199):

if (requestJson.has("system")
        && !requestJson.get("system").isNull()
        && !requestJson.get("system").asText().isEmpty()) {
    var systemNode = BraintrustJsonMapper.get().createObjectNode();
    systemNode.put("role", "system");
    systemNode.set("content", requestJson.get("system"));
    inputArray.add(systemNode);
}

The problem is requestJson.get("system").asText() — for a JSON array node, Jackson's asText() returns "" (empty string). The .isEmpty() check then evaluates to true, and the entire block is skipped. The systemNode.set("content", ...) on line 197 would correctly handle both string and array values, but it is never reached.

Fix

Replace the string-based emptiness check with a type-aware check:

if (requestJson.has("system") && !requestJson.get("system").isNull()) {
    JsonNode systemValue = requestJson.get("system");
    boolean isEmpty = systemValue.isTextual() && systemValue.asText().isEmpty();
    if (!isEmpty) {
        var systemNode = BraintrustJsonMapper.get().createObjectNode();
        systemNode.put("role", "system");
        systemNode.set("content", systemValue);
        inputArray.add(systemNode);
    }
}
Impact

When a user enables Anthropic prompt caching on their system prompt (a key cost-optimization feature), the system prompt disappears from the Braintrust span input. The trace shows the user messages but no system instructions, making it impossible to understand the model's behavior from the trace alone.

This affects:

  • Direct Anthropic SDK calls via BraintrustAnthropic.wrapAnthropic() with system in array format
  • Any request using cache_control on system prompt blocks

Non-streaming and streaming calls are both affected (the request tagging is shared).

Braintrust docs status

Upstream sources

Local files inspected

  • braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java — lines 192–199 (tagAnthropicRequest: asText() on array returns "", causing skip)
  • braintrust-sdk/instrumentation/anthropic_2_2_0/src/main/java/dev/braintrust/instrumentation/anthropic/v2_2_0/TracingHttpClient.java — delegates to InstrumentationSemConv
  • braintrust-sdk/instrumentation/anthropic_2_2_0/src/test/java/dev/braintrust/instrumentation/anthropic/v2_2_0/BraintrustAnthropicTest.java — no test uses array-format system prompts
Dominant language
Java
Stars
21
Forks
5
Avg merge
2d 7h
Merged PRs (30d)
8

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from braintrustdata/braintrust-sdk-java

All issues in braintrustdata/braintrust-sdk-java

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.