[BOT ISSUE] Anthropic system prompt silently dropped from span input when using array format (prompt caching)
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 82/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Quiet
- Tech stack
- java
- Domain
- observability
Research direction
Start in braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java at tagAnthropicRequest(), then inspect BraintrustAnthropicTest.java for existing request-tagging coverage. Verify that string and array-format system prompts are retained in braintrust.input_json, including the array format used with cache_control, for both shared streaming and non-streaming paths.
Written by the indexing model from the issue text.
Description
Summary
The Anthropic request tagger in InstrumentationSemConv.tagAnthropicRequest() silently drops the system prompt from braintrust.input_json when it is provided in array-of-content-blocks format. This array format is required when using Anthropic prompt caching on system prompts (each block needs a cache_control marker), and is explicitly documented by both Anthropic and Braintrust.
When system is a plain string (e.g. "system": "You are helpful"), it is correctly appended to the input array as a synthetic {role: "system", content: "..."} entry. But when system is an array (e.g. "system": [{"type":"text","text":"You are helpful","cache_control":{"type":"ephemeral"}}]), the guard condition fails and the system prompt is silently omitted.
What is missing
In InstrumentationSemConv.tagAnthropicRequest() (lines 192–199):
if (requestJson.has("system")
&& !requestJson.get("system").isNull()
&& !requestJson.get("system").asText().isEmpty()) {
var systemNode = BraintrustJsonMapper.get().createObjectNode();
systemNode.put("role", "system");
systemNode.set("content", requestJson.get("system"));
inputArray.add(systemNode);
}
The problem is requestJson.get("system").asText() — for a JSON array node, Jackson's asText() returns "" (empty string). The .isEmpty() check then evaluates to true, and the entire block is skipped. The systemNode.set("content", ...) on line 197 would correctly handle both string and array values, but it is never reached.
Fix
Replace the string-based emptiness check with a type-aware check:
if (requestJson.has("system") && !requestJson.get("system").isNull()) {
JsonNode systemValue = requestJson.get("system");
boolean isEmpty = systemValue.isTextual() && systemValue.asText().isEmpty();
if (!isEmpty) {
var systemNode = BraintrustJsonMapper.get().createObjectNode();
systemNode.put("role", "system");
systemNode.set("content", systemValue);
inputArray.add(systemNode);
}
}
Impact
When a user enables Anthropic prompt caching on their system prompt (a key cost-optimization feature), the system prompt disappears from the Braintrust span input. The trace shows the user messages but no system instructions, making it impossible to understand the model's behavior from the trace alone.
This affects:
- Direct Anthropic SDK calls via
BraintrustAnthropic.wrapAnthropic()withsystemin array format - Any request using
cache_controlon system prompt blocks
Non-streaming and streaming calls are both affected (the request tagging is shared).
Braintrust docs status
- Braintrust docs at https://www.braintrust.dev/docs/integrations/ai-providers/anthropic explicitly document prompt caching support: "When you use prompt caching, Braintrust automatically captures cache read and creation token counts as span metrics." The code examples show the array system format with
cache_control: supported
Upstream sources
- Anthropic prompt caching docs: https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching — documents
systemas array of content blocks withcache_controlmarkers as the standard pattern for caching system prompts - Anthropic Messages API: https://docs.anthropic.com/en/api/messages —
systemfield accepts both string andArray<ContentBlockParam>formats - Anthropic Java SDK:
MessageCreateParams.system()accepts bothStringandList<TextBlockParam>— both route through the same HTTP client
Local files inspected
braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java— lines 192–199 (tagAnthropicRequest:asText()on array returns"", causing skip)braintrust-sdk/instrumentation/anthropic_2_2_0/src/main/java/dev/braintrust/instrumentation/anthropic/v2_2_0/TracingHttpClient.java— delegates toInstrumentationSemConvbraintrust-sdk/instrumentation/anthropic_2_2_0/src/test/java/dev/braintrust/instrumentation/anthropic/v2_2_0/BraintrustAnthropicTest.java— no test uses array-format system prompts
- Dominant language
- Java
- Stars
- 21
- Forks
- 5
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 8
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from braintrustdata/braintrust-sdk-java
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
All issues in braintrustdata/braintrust-sdk-java
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
infinispan/infinispan#18150 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
untriaged
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
opensearch-project/k-NN#3597 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100