Workers AI integration drops output of non-streaming chat responses in the OpenAI format (`choices`)

Abierto Apto para principiantes
  • ¿Algo no está bien?
#26,120 1 comentario 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
2/5
Tiempo estimado
1-3 horas
Aptitud para principiantes
78/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
typescript
Área
observability

Línea de trabajo

Empieza leyendo packages/server-utils/src/ai/workers-ai/utils.ts y su función addResponseAttributes; después, compara el manejo actual del streaming de Workers AI en ai/workers-ai/streaming.ts con el de OpenAI en ai/openai/utils.ts. Actualiza la ruta sin streaming para capturar los mensajes de salida, el texto de la respuesta y los motivos de finalización de choices; confirma que la respuesta en el nuevo formato registre esos atributos y conserve el formato de respuesta anterior.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Bug Cloudflare Workers Spans
Is there an existing issue for this?
  • I have checked for existing issues
How do you use Sentry?

Sentry SaaS (sentry.io)

Which SDK are you using?

@sentry/cloudflare

SDK Version

11.10.0 (@sentry/server-utils 11.10.0). The same code is on develop.

Framework Version

Cloudflare Workers, Workers AI binding

Steps to Reproduce

  1. Instrument the Workers AI binding (the default @sentry/cloudflare setup).

  2. Call a model that answers in the OpenAI chat-completion format, without stream: true. For example, @cf/google/gemma-4-26b-a4b-it:

    const result = await env.AI.run("@cf/google/gemma-4-26b-a4b-it", {
  messages: [{ role: "user", content: "Say hi" }],
  max_tokens: 50,
});

  3. Open the gen_ai.chat span in Sentry.

Expected Result

The span has gen_ai.output.messages, gen_ai.response.text and gen_ai.response.finish_reasons, the same as for a stream: true call to the same model.

Actual Result

The span has the token usage, but no output and no finish reason.

I sent the same prompt to @cf/google/gemma-4-26b-a4b-it twice through the Workers AI REST API, once without and once with stream: true. Then I passed the real responses through instrumentWorkersAiClient (11.10.0). The gen_ai.* span attributes were:

--- no stream
gen_ai.operation.name = "chat"
gen_ai.usage.input_tokens = 19
gen_ai.usage.output_tokens = 4
gen_ai.usage.total_tokens = 23

--- stream: true
gen_ai.operation.name = "chat"
gen_ai.output.messages = "[{\"role\":\"assistant\",\"parts\":[{\"type\":\"text\",\"content\":\"Hello there!\"}]}]"
gen_ai.response.finish_reasons = "[\"stop\"]"
gen_ai.response.streaming = true
gen_ai.response.text = "Hello there!"
gen_ai.usage.input_tokens = 19
gen_ai.usage.output_tokens = 4
gen_ai.usage.total_tokens = 23

The non-streaming response was:

{"id":"...","object":"chat.completion","created":0,"model":"@cf/google/gemma-4-26b-a4b-it","choices":[{"index":0,"message":{"role":"assistant","content":"Hello there!"},"finish_reason":"stop","logprobs":null}],"usage":{"prompt_tokens":19,"completion_tokens":4,"total_tokens":23}}
Cause

Workers AI returns two response formats:

  • older models: { response: string, tool_calls?, usage }
  • newer models, such as gemma-4: the OpenAI chat-completion format with choices[].message

The streaming path supports both. instrumentWorkersAiStream in ai/workers-ai/streaming.ts reads parsed.response and also parsed.choices[].delta.content, delta.tool_calls and finish_reason.

The non-streaming path supports only the old format. addResponseAttributes in ai/workers-ai/utils.ts reads only response.response and response.tool_calls:

https://github.com/getsentry/sentry-javascript/blob/develop/packages/server-utils/src/ai/workers-ai/utils.ts

A possible fix: when response.choices is an array, read choices[].message.content, choices[].message.tool_calls and choices[].finish_reason, the same way as the OpenAI integration (ai/openai/utils.ts) does.

Additional Context

Workaround: we wrap the call in our own gen_ai.chat span that sets gen_ai.output.messages from choices, and we call env.AI.run inside Sentry.suppressTracing() so that the SDK span does not duplicate it.

Lenguaje dominante
TypeScript
Estrellas
8.8k
Forks
1.9k
Merge medio
1 d 20 h
PR fusionados (30 d)
682

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de getsentry/sentry-javascript

Todos los issues de getsentry/sentry-javascript

Issues similares

Más issues de TypeScript

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.