[BOT ISSUE] OpenAI embeddings spans missing output capture and incomplete input extraction

Abierto Apto para principiantes
#66 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
2/5
Tiempo estimado
1-3 horas
Aptitud para principiantes
68/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Tranquilo
Stack tecnológico
java

Línea de trabajo

Empieza en braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java revisando tagOpenAIRequest(), tagOpenAIResponse() y el mapeo del span de Embeddings cerca de la línea 244. Comprueba BraintrustOpenAITest.java y las formas de la OpenAI embeddings API; se considera terminado cuando las entradas de una sola cadena y las respuestas data[].embedding rellenan los datos del span sin introducir regresiones en el manejo de Chat Completions o Responses.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Summary

The OpenAI instrumentation recognizes embeddings calls (span is named "Embeddings") but the span data is largely empty because the response and request parsing logic in InstrumentationSemConv only handles Chat Completions and Responses API field shapes.

Specifically:

  • No output_json: the response parser checks for choices (Chat Completions) or output (Responses API), but embeddings responses have data[].embedding — neither branch matches
  • Incomplete input_json: the request parser checks for messages or input (as array), but embeddings input can be a single string, which fails the isArray() check on line 102
  • Partial metrics: prompt_tokens and total_tokens are captured correctly from usage, but completion_tokens is absent (embeddings don't produce completion tokens) — this is fine but worth noting

What is missing

In InstrumentationSemConv.tagOpenAIRequest() (lines 78–108):

if (requestJson.has("messages")) {
    span.setAttribute("braintrust.input_json", toJson(requestJson.get("messages")));
} else if (requestJson.has("input") && requestJson.get("input").isArray()) {
    span.setAttribute("braintrust.input_json", toJson(requestJson.get("input")));
}
  • Embeddings requests use input which can be a string ("Hello world"), an array of strings, or an array of token arrays. Single-string inputs fail the isArray() guard and are silently dropped.

In InstrumentationSemConv.tagOpenAIResponse() (lines 111–156):

if (responseJson.has("choices")) {
    span.setAttribute("braintrust.output_json", toJson(responseJson.get("choices")));
} else if (responseJson.has("output")) {
    span.setAttribute("braintrust.output_json", toJson(responseJson.get("output")));
}
  • Embeddings responses have data (array of embedding objects) and model — neither choices nor output. No output_json is set.

Additionally, the model field from the embeddings request (e.g., text-embedding-3-small) is correctly extracted into metadata, and usage metrics partially work — so this is a detail gap rather than total failure.

Braintrust docs status

  • The Braintrust OpenAI integration docs at braintrust.dev/docs/integrations/ai-providers/openai do not mention embeddings: not_found
  • No embeddings instrumentation is documented for any provider

Upstream sources

Local files inspected

  • braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java — lines 78–108 (tagOpenAIRequest: input string case not handled), lines 111–156 (tagOpenAIResponse: no data field extraction), line 244 (span name correctly maps to "Embeddings")
  • braintrust-sdk/instrumentation/openai_2_8_0/src/main/java/dev/braintrust/instrumentation/openai/v2_8_0/TracingHttpClient.java — HTTP-level wrapping captures the call, delegates to InstrumentationSemConv
  • braintrust-sdk/instrumentation/openai_2_8_0/src/test/java/dev/braintrust/instrumentation/openai/v2_8_0/BraintrustOpenAITest.java — no embeddings test exists
Lenguaje dominante
Java
Estrellas
21
Forks
5
Merge medio
2 d 7 h
PR fusionados (30 d)
8

Guía de contribución

Abrir la guía de contribución

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de braintrustdata/braintrust-sdk-java

Todos los issues de braintrustdata/braintrust-sdk-java

Issues similares

Más issues de Java

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.