Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

otelMiddleware: captureContent drops image/audio/video/document parts, recording only '[image]' placeholders

Closed
  • Is something wrong?
#1,525 0 comments 0 reactions 1 assignee View on GitHub

Maintainers usually reply within 1 day

@AlemTuzlak is already working on this.

Since Sep 27, 2026.

  • #1527 by @tombeckenham — open

Assessment

This issue has not been assessed yet.

Description

has-pr waiting-on: maintainer

Summary

With captureContent: true, otelMiddleware replaces every non-text content part with a placeholder string before writing gen_ai.input.messages / langfuse.observation.input. A vision call's recorded input reads:

[{"role":"user","content":"Describe this reference image. [image]"}]

The image is gone from the trace. For vision classification this is the part you most need when debugging. For example: why did the model call this image an illustration? You can see the model's answer but not what it looked at.

Root cause

serializeContent (packages/ai/src/middlewares/otel.ts:185) flattens each message to one string. Image, audio, video and document parts become '[image]', '[audio]' and so on, and source is discarded:

case 'image':
  parts.push('[image]')
  break

The result is written as { role, content: string } (line ~584), so there is nowhere to put a structured part even if one survived.

There is no way to work around this from outside the middleware:

  • attributeEnricher runs before the captureContent block, so anything it sets on gen_ai.input.messages gets overwritten.
  • redact only ever sees the already-flattened string.

Proposal

Keep content parts structured in gen_ai.input.messages, following the OTel GenAI semconv parts shape, which PostHog, Langfuse and Datadog all render:

  • text → { type: 'text', content }
  • image / audio / video / document with source.type === 'url' → { type: 'uri', modality, uri: source.value, mime_type? }
  • source.type === 'data' (inline base64) → keep the placeholder by default, since it would blow maxContentLength and span attribute limits. Optionally add an opt-in hook (e.g. serializePart?: (part) => unknown) so an app can upload the blob and substitute a URL.

redact should still apply to the text parts.

If changing the attribute shape is too breaking for 0.x, an opt-in structuredContent: true flag would also work.

Version

@tanstack/ai 0.58.0 (current main has the same code).

I'm happy to open a PR.

Dominant language
TypeScript
Stars
3.1k
Forks
340
Avg merge
2d 10h
Merged PRs (30d)
174

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from TanStack/ai

All issues in TanStack/ai

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.