[BOT ISSUE] Anthropic system prompt silently dropped from span input when using array format (prompt caching)
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 2/5
- Tiempo estimado
- 1-3 horas
- Aptitud para principiantes
- 82/100
- Tipo de issue
- Error
- Claridad
- Bien especificado
- Estado de actividad
- Tranquilo
- Stack tecnológico
- java
- Área
- observability
Línea de trabajo
Comienza en braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java, en tagAnthropicRequest(), y después inspecciona BraintrustAnthropicTest.java para comprobar la cobertura existente del etiquetado de requests. Verifica que los prompts de sistema en formato string y array se conserven en braintrust.input_json, incluido el formato array usado con cache_control, tanto para las rutas compartidas de streaming como para las que no usan streaming.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Summary
The Anthropic request tagger in InstrumentationSemConv.tagAnthropicRequest() silently drops the system prompt from braintrust.input_json when it is provided in array-of-content-blocks format. This array format is required when using Anthropic prompt caching on system prompts (each block needs a cache_control marker), and is explicitly documented by both Anthropic and Braintrust.
When system is a plain string (e.g. "system": "You are helpful"), it is correctly appended to the input array as a synthetic {role: "system", content: "..."} entry. But when system is an array (e.g. "system": [{"type":"text","text":"You are helpful","cache_control":{"type":"ephemeral"}}]), the guard condition fails and the system prompt is silently omitted.
What is missing
In InstrumentationSemConv.tagAnthropicRequest() (lines 192–199):
if (requestJson.has("system")
&& !requestJson.get("system").isNull()
&& !requestJson.get("system").asText().isEmpty()) {
var systemNode = BraintrustJsonMapper.get().createObjectNode();
systemNode.put("role", "system");
systemNode.set("content", requestJson.get("system"));
inputArray.add(systemNode);
}
The problem is requestJson.get("system").asText() — for a JSON array node, Jackson's asText() returns "" (empty string). The .isEmpty() check then evaluates to true, and the entire block is skipped. The systemNode.set("content", ...) on line 197 would correctly handle both string and array values, but it is never reached.
Fix
Replace the string-based emptiness check with a type-aware check:
if (requestJson.has("system") && !requestJson.get("system").isNull()) {
JsonNode systemValue = requestJson.get("system");
boolean isEmpty = systemValue.isTextual() && systemValue.asText().isEmpty();
if (!isEmpty) {
var systemNode = BraintrustJsonMapper.get().createObjectNode();
systemNode.put("role", "system");
systemNode.set("content", systemValue);
inputArray.add(systemNode);
}
}
Impact
When a user enables Anthropic prompt caching on their system prompt (a key cost-optimization feature), the system prompt disappears from the Braintrust span input. The trace shows the user messages but no system instructions, making it impossible to understand the model's behavior from the trace alone.
This affects:
- Direct Anthropic SDK calls via
BraintrustAnthropic.wrapAnthropic()withsystemin array format - Any request using
cache_controlon system prompt blocks
Non-streaming and streaming calls are both affected (the request tagging is shared).
Braintrust docs status
- Braintrust docs at https://www.braintrust.dev/docs/integrations/ai-providers/anthropic explicitly document prompt caching support: "When you use prompt caching, Braintrust automatically captures cache read and creation token counts as span metrics." The code examples show the array system format with
cache_control: supported
Upstream sources
- Anthropic prompt caching docs: https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching — documents
systemas array of content blocks withcache_controlmarkers as the standard pattern for caching system prompts - Anthropic Messages API: https://docs.anthropic.com/en/api/messages —
systemfield accepts both string andArray<ContentBlockParam>formats - Anthropic Java SDK:
MessageCreateParams.system()accepts bothStringandList<TextBlockParam>— both route through the same HTTP client
Local files inspected
braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java— lines 192–199 (tagAnthropicRequest:asText()on array returns"", causing skip)braintrust-sdk/instrumentation/anthropic_2_2_0/src/main/java/dev/braintrust/instrumentation/anthropic/v2_2_0/TracingHttpClient.java— delegates toInstrumentationSemConvbraintrust-sdk/instrumentation/anthropic_2_2_0/src/test/java/dev/braintrust/instrumentation/anthropic/v2_2_0/BraintrustAnthropicTest.java— no test uses array-format system prompts
- Lenguaje dominante
- Java
- Estrellas
- 21
- Forks
- 5
- Merge medio
- 2 d 7 h
- PR fusionados (30 d)
- 8
Guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de braintrustdata/braintrust-sdk-java
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
-
Dificultad 5/5 Más de una semana Aptitud para principiantes 35/100
Todos los issues de braintrustdata/braintrust-sdk-java
Issues similares
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
infinispan/infinispan#18150 ·
-
area/frontend
Dificultad 2/5 1-3 horas Aptitud para principiantes 65/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 84/100
-
untriaged
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
opensearch-project/k-NN#3597 ·
-
bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 88/100