Workers AI integration drops output of non-streaming chat responses in the OpenAI format (`choices`)

Offen Anfängerfreundlich
  • Stimmt etwas nicht?
#26,120 1 Kommentar 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
2/5
Geschätzter Aufwand
1-3 Stunden
Anfängerfreundlichkeit
78/100
Issue-Typ
Bug
Klarheit
Klar beschrieben
Aktivitätsstatus
Aktiv
Tech-Stack
typescript
Bereich
observability

Rechercherichtung

Lies zuerst packages/server-utils/src/ai/workers-ai/utils.ts und die Funktion addResponseAttributes und vergleiche anschließend die bestehende Workers-AI-Streamverarbeitung in ai/workers-ai/streaming.ts mit der OpenAI-Verarbeitung in ai/openai/utils.ts. Aktualisiere den Nicht-Streaming-Pfad so, dass Ausgabenachrichten, Antworttext und Beendigungsgründe aus choices erfasst werden; bestätige, dass die Antwort im neuen Format diese Attribute aufzeichnet und das ältere Antwortformat beibehält.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

Bug Cloudflare Workers Spans
Is there an existing issue for this?
  • I have checked for existing issues
How do you use Sentry?

Sentry SaaS (sentry.io)

Which SDK are you using?

@sentry/cloudflare

SDK Version

11.10.0 (@sentry/server-utils 11.10.0). The same code is on develop.

Framework Version

Cloudflare Workers, Workers AI binding

Steps to Reproduce

  1. Instrument the Workers AI binding (the default @sentry/cloudflare setup).

  2. Call a model that answers in the OpenAI chat-completion format, without stream: true. For example, @cf/google/gemma-4-26b-a4b-it:

    const result = await env.AI.run("@cf/google/gemma-4-26b-a4b-it", {
  messages: [{ role: "user", content: "Say hi" }],
  max_tokens: 50,
});

  3. Open the gen_ai.chat span in Sentry.

Expected Result

The span has gen_ai.output.messages, gen_ai.response.text and gen_ai.response.finish_reasons, the same as for a stream: true call to the same model.

Actual Result

The span has the token usage, but no output and no finish reason.

I sent the same prompt to @cf/google/gemma-4-26b-a4b-it twice through the Workers AI REST API, once without and once with stream: true. Then I passed the real responses through instrumentWorkersAiClient (11.10.0). The gen_ai.* span attributes were:

--- no stream
gen_ai.operation.name = "chat"
gen_ai.usage.input_tokens = 19
gen_ai.usage.output_tokens = 4
gen_ai.usage.total_tokens = 23

--- stream: true
gen_ai.operation.name = "chat"
gen_ai.output.messages = "[{\"role\":\"assistant\",\"parts\":[{\"type\":\"text\",\"content\":\"Hello there!\"}]}]"
gen_ai.response.finish_reasons = "[\"stop\"]"
gen_ai.response.streaming = true
gen_ai.response.text = "Hello there!"
gen_ai.usage.input_tokens = 19
gen_ai.usage.output_tokens = 4
gen_ai.usage.total_tokens = 23

The non-streaming response was:

{"id":"...","object":"chat.completion","created":0,"model":"@cf/google/gemma-4-26b-a4b-it","choices":[{"index":0,"message":{"role":"assistant","content":"Hello there!"},"finish_reason":"stop","logprobs":null}],"usage":{"prompt_tokens":19,"completion_tokens":4,"total_tokens":23}}
Cause

Workers AI returns two response formats:

  • older models: { response: string, tool_calls?, usage }
  • newer models, such as gemma-4: the OpenAI chat-completion format with choices[].message

The streaming path supports both. instrumentWorkersAiStream in ai/workers-ai/streaming.ts reads parsed.response and also parsed.choices[].delta.content, delta.tool_calls and finish_reason.

The non-streaming path supports only the old format. addResponseAttributes in ai/workers-ai/utils.ts reads only response.response and response.tool_calls:

https://github.com/getsentry/sentry-javascript/blob/develop/packages/server-utils/src/ai/workers-ai/utils.ts

A possible fix: when response.choices is an array, read choices[].message.content, choices[].message.tool_calls and choices[].finish_reason, the same way as the OpenAI integration (ai/openai/utils.ts) does.

Additional Context

Workaround: we wrap the call in our own gen_ai.chat span that sets gen_ai.output.messages from choices, and we call env.AI.run inside Sentry.suppressTracing() so that the SDK span does not duplicate it.

Vorherrschende Sprache
TypeScript
Sterne
8.8k
Forks
1.9k
Ø Merge
1 T. 20 Std.
Gemergte PRs (30 T.)
682

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus getsentry/sentry-javascript

Alle Issues in getsentry/sentry-javascript

Ähnliche Issues

Weitere Issues zu TypeScript

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.