Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

otelMiddleware: captureContent drops image/audio/video/document parts, recording only '[image]' placeholders

Cerrado
  • ¿Algo no está bien?
#1,525 0 comentarios 0 reacciones 1 asignado Ver en GitHub

Los mantenedores suelen responder en 1 día

@AlemTuzlak ya está trabajando en esto.

Desde el 27/9/2026.

  • #1527 de @tombeckenham — abierto

Evaluación

Este issue todavía no se ha evaluado.

Descripción

has-pr waiting-on: maintainer

Summary

With captureContent: true, otelMiddleware replaces every non-text content part with a placeholder string before writing gen_ai.input.messages / langfuse.observation.input. A vision call's recorded input reads:

[{"role":"user","content":"Describe this reference image. [image]"}]

The image is gone from the trace. For vision classification this is the part you most need when debugging. For example: why did the model call this image an illustration? You can see the model's answer but not what it looked at.

Root cause

serializeContent (packages/ai/src/middlewares/otel.ts:185) flattens each message to one string. Image, audio, video and document parts become '[image]', '[audio]' and so on, and source is discarded:

case 'image':
  parts.push('[image]')
  break

The result is written as { role, content: string } (line ~584), so there is nowhere to put a structured part even if one survived.

There is no way to work around this from outside the middleware:

  • attributeEnricher runs before the captureContent block, so anything it sets on gen_ai.input.messages gets overwritten.
  • redact only ever sees the already-flattened string.

Proposal

Keep content parts structured in gen_ai.input.messages, following the OTel GenAI semconv parts shape, which PostHog, Langfuse and Datadog all render:

  • text → { type: 'text', content }
  • image / audio / video / document with source.type === 'url' → { type: 'uri', modality, uri: source.value, mime_type? }
  • source.type === 'data' (inline base64) → keep the placeholder by default, since it would blow maxContentLength and span attribute limits. Optionally add an opt-in hook (e.g. serializePart?: (part) => unknown) so an app can upload the blob and substitute a URL.

redact should still apply to the text parts.

If changing the attribute shape is too breaking for 0.x, an opt-in structuredContent: true flag would also work.

Version

@tanstack/ai 0.58.0 (current main has the same code).

I'm happy to open a PR.

Lenguaje dominante
TypeScript
Estrellas
3.1k
Forks
340
Merge medio
2 d 7 h
PR fusionados (30 d)
175

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de TanStack/ai

Todos los issues de TanStack/ai

Issues similares

Más issues de TypeScript

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.