[bot] Cloudflare Workers AI `queueRequest` (batch) inference is explicitly excluded from tracing

Aperta
  • Qualcosa non va?
#2,684 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
30/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
typescript
Ambito
observability

Direzione di ricerca

Start in js/src/instrumentation/plugins/workers-ai-plugin.ts: the shouldTrace predicate at lines 111-116 currently rejects queueRequest, while DECISION_MODELS and the extraction check define the already-supported models. Read js/src/vendor-sdk-types/workers-ai.ts for the typed options and js/src/instrumentation/plugins/workers-ai-interceptors.ts for the run channel, then compare how existing traced run calls create spans. Done means queued submissions and later request_id polling produce trace spans with request/response and usage, aligned with the documented Cloudflare batch flow.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

cloudflare typescript

Summary

Cloudflare Workers AI's env.AI.run() binding supports an official asynchronous/batch inference mode via queueRequest: true (submit returns {status: "queued", request_id}; the same request shape is later resubmitted with only {request_id} to poll/retrieve results). This repo's Workers AI instrumentation explicitly detects this option and skips tracing whenever it is set — the exclusion is coded and typed, not an oversight, and it is also the one case in this audit where Braintrust's own published docs state outright that these calls are not traced.

What's missing

In js/src/instrumentation/plugins/workers-ai-plugin.ts, the shouldTrace predicate for the single run channel (lines 111-116) is:

shouldTrace: () =>
  (extraction || DECISION_MODELS.has(args[0])) &&
  !args[1]?.stream &&
  !args[2]?.returnRawResponse &&
  !args[2]?.websocket &&
  !args[2]?.queueRequest,

with the adjacent comment (lines 109-110): "These models return JSON directly. Do not consume raw bodies or claim to trace an asynchronous submission or unsupported stream." So:

  • Any call with queueRequest: true (the initial async submission, and the later polling call) is unconditionally skipped — no span, no request/response capture, no usage metrics — even though the predicate's other condition (DECISION_MODELS/extraction model allow-list, lines 12-16 and line 101) is already satisfied.
  • WorkersAIOptions in js/src/vendor-sdk-types/workers-ai.ts (lines 30-35) explicitly types queueRequest?: boolean alongside returnRawResponse?: boolean and websocket?: boolean, confirming this is a deliberately recognized, typed option rather than an unknown/unsupported shape.
  • js/src/wrappers/workers-ai.ts (JSDoc, lines 6-18) documents the behavior directly: "Trace Clef and Clairic calls on a Workers AI binding. Other models and raw-response/streaming modes pass through unchanged." — batch/queued calls pass through completely untraced.

Note: Workers AI tracing in this repo is already narrowly scoped to the DECISION_MODELS set (@cf/cloudflare/clef, clef-flash, clef-omni) plus the clairic extraction model — this gap is specifically about those same already-in-scope models losing all tracing the moment a caller uses the batch/queue submission mode that Cloudflare officially supports for Workers AI inference generally.

Braintrust docs status

Explicitly documented as excluded (stronger than not_found). https://www.braintrust.dev/docs/integrations/sdk-integrations/cloudflare.md states: "Calls to other models, and calls that stream, return a raw response, use a WebSocket, or queue the request, pass through unchanged," and instructs users to "wrap the AI binding with wrapWorkersAI() and call run() on the wrapped binding." Braintrust's own docs affirmatively tell users that queued (batch) Workers AI calls will not be traced.

Upstream sources

Braintrust docs sources checked

Local repo files inspected

  • js/src/instrumentation/plugins/workers-ai-plugin.ts — DECISION_MODELS (lines 12-16), shouldTrace predicate (lines 111-116), extraction model check (line 101)
  • js/src/vendor-sdk-types/workers-ai.ts — WorkersAIOptions (lines 30-35), including the typed queueRequest?: boolean field
  • js/src/wrappers/workers-ai.ts — JSDoc (lines 6-18) and wrapWorkersAI() implementation
  • js/src/instrumentation/plugins/workers-ai-interceptors.ts — workersAIChannels.run channel definition
Lingua principale
TypeScript
Stelle
29
Fork
22
Merge medio
1g 6h
PR unite (30g)
69

Preparare l'ambiente

  • Include un Dockerfile o un file Docker Compose
  • Nessun modello di pull request
  • Nessuna guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di braintrustdata/braintrust-sdk-javascript

Tutte le issue di braintrustdata/braintrust-sdk-javascript

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.