Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

otelMiddleware: captureContent drops image/audio/video/document parts, recording only '[image]' placeholders

Chiusa
  • Qualcosa non va?
#1,525 0 commenti 0 reazioni 1 assegnatario Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@AlemTuzlak ci sta già lavorando.

Dal 27/9/2026.

  • #1527 di @tombeckenham — aperta

Valutazione

Questa issue non è ancora stata valutata.

Descrizione

has-pr waiting-on: maintainer

Summary

With captureContent: true, otelMiddleware replaces every non-text content part with a placeholder string before writing gen_ai.input.messages / langfuse.observation.input. A vision call's recorded input reads:

[{"role":"user","content":"Describe this reference image. [image]"}]

The image is gone from the trace. For vision classification this is the part you most need when debugging. For example: why did the model call this image an illustration? You can see the model's answer but not what it looked at.

Root cause

serializeContent (packages/ai/src/middlewares/otel.ts:185) flattens each message to one string. Image, audio, video and document parts become '[image]', '[audio]' and so on, and source is discarded:

case 'image':
  parts.push('[image]')
  break

The result is written as { role, content: string } (line ~584), so there is nowhere to put a structured part even if one survived.

There is no way to work around this from outside the middleware:

  • attributeEnricher runs before the captureContent block, so anything it sets on gen_ai.input.messages gets overwritten.
  • redact only ever sees the already-flattened string.

Proposal

Keep content parts structured in gen_ai.input.messages, following the OTel GenAI semconv parts shape, which PostHog, Langfuse and Datadog all render:

  • text → { type: 'text', content }
  • image / audio / video / document with source.type === 'url' → { type: 'uri', modality, uri: source.value, mime_type? }
  • source.type === 'data' (inline base64) → keep the placeholder by default, since it would blow maxContentLength and span attribute limits. Optionally add an opt-in hook (e.g. serializePart?: (part) => unknown) so an app can upload the blob and substitute a URL.

redact should still apply to the text parts.

If changing the attribute shape is too breaking for 0.x, an opt-in structuredContent: true flag would also work.

Version

@tanstack/ai 0.58.0 (current main has the same code).

I'm happy to open a PR.

Lingua principale
TypeScript
Stelle
3.1k
Fork
340
Merge medio
2g 10h
PR unite (30g)
169

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di TanStack/ai

Tutte le issue di TanStack/ai

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.