[BOT ISSUE] Anthropic system prompt silently dropped from span input when using array format (prompt caching)

Abierto Apto para principiantes
#81 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
2/5
Tiempo estimado
1-3 horas
Aptitud para principiantes
82/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Tranquilo
Stack tecnológico
java

Línea de trabajo

Comienza en braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java, en tagAnthropicRequest(), y después inspecciona BraintrustAnthropicTest.java para comprobar la cobertura existente del etiquetado de requests. Verifica que los prompts de sistema en formato string y array se conserven en braintrust.input_json, incluido el formato array usado con cache_control, tanto para las rutas compartidas de streaming como para las que no usan streaming.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Summary

The Anthropic request tagger in InstrumentationSemConv.tagAnthropicRequest() silently drops the system prompt from braintrust.input_json when it is provided in array-of-content-blocks format. This array format is required when using Anthropic prompt caching on system prompts (each block needs a cache_control marker), and is explicitly documented by both Anthropic and Braintrust.

When system is a plain string (e.g. "system": "You are helpful"), it is correctly appended to the input array as a synthetic {role: "system", content: "..."} entry. But when system is an array (e.g. "system": [{"type":"text","text":"You are helpful","cache_control":{"type":"ephemeral"}}]), the guard condition fails and the system prompt is silently omitted.

What is missing

In InstrumentationSemConv.tagAnthropicRequest() (lines 192–199):

if (requestJson.has("system")
        && !requestJson.get("system").isNull()
        && !requestJson.get("system").asText().isEmpty()) {
    var systemNode = BraintrustJsonMapper.get().createObjectNode();
    systemNode.put("role", "system");
    systemNode.set("content", requestJson.get("system"));
    inputArray.add(systemNode);
}

The problem is requestJson.get("system").asText() — for a JSON array node, Jackson's asText() returns "" (empty string). The .isEmpty() check then evaluates to true, and the entire block is skipped. The systemNode.set("content", ...) on line 197 would correctly handle both string and array values, but it is never reached.

Fix

Replace the string-based emptiness check with a type-aware check:

if (requestJson.has("system") && !requestJson.get("system").isNull()) {
    JsonNode systemValue = requestJson.get("system");
    boolean isEmpty = systemValue.isTextual() && systemValue.asText().isEmpty();
    if (!isEmpty) {
        var systemNode = BraintrustJsonMapper.get().createObjectNode();
        systemNode.put("role", "system");
        systemNode.set("content", systemValue);
        inputArray.add(systemNode);
    }
}
Impact

When a user enables Anthropic prompt caching on their system prompt (a key cost-optimization feature), the system prompt disappears from the Braintrust span input. The trace shows the user messages but no system instructions, making it impossible to understand the model's behavior from the trace alone.

This affects:

  • Direct Anthropic SDK calls via BraintrustAnthropic.wrapAnthropic() with system in array format
  • Any request using cache_control on system prompt blocks

Non-streaming and streaming calls are both affected (the request tagging is shared).

Braintrust docs status

Upstream sources

Local files inspected

  • braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java — lines 192–199 (tagAnthropicRequest: asText() on array returns "", causing skip)
  • braintrust-sdk/instrumentation/anthropic_2_2_0/src/main/java/dev/braintrust/instrumentation/anthropic/v2_2_0/TracingHttpClient.java — delegates to InstrumentationSemConv
  • braintrust-sdk/instrumentation/anthropic_2_2_0/src/test/java/dev/braintrust/instrumentation/anthropic/v2_2_0/BraintrustAnthropicTest.java — no test uses array-format system prompts
Lenguaje dominante
Java
Estrellas
21
Forks
5
Merge medio
2 d 7 h
PR fusionados (30 d)
8

Guía de contribución

Abrir la guía de contribución

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de braintrustdata/braintrust-sdk-java

Todos los issues de braintrustdata/braintrust-sdk-java

Issues similares

Más issues de Java

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.