Hacktoberfest 2026：维护者为十月标记出来的 issue，仍然开放、适合新手。 浏览 Hacktoberfest issue

Google GenAI `generateContentStream` aggregation silently drops `inlineData` output parts (images, audio)

已关闭 适合新手
  • 哪里不对？
#1,690 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
2/5
预计耗时
1-3 小时
新手友好度
68/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
冷清
技术栈
typescript
领域
observability

调研方向

从 js/src/instrumentation/plugins/google-genai-plugin.ts 中的 aggregateGenerateContentChunks 开始，将其对 part 的处理与 js/src/vendor-sdk-types/google-genai.ts 中的 inlineData 声明进行比较。检查 e2e/scenarios/google-genai-instrumentation/ 中现有的 instrumentation 检查。完成标准是：流式传输的 inlineData 部分会保留在聚合输出中，并且有针对图像或其他二进制输出的覆盖。

由索引模型根据 Issue 内容生成。

描述

stale

Summary

The Google GenAI streaming aggregation code drops inlineData parts from the response output. When Gemini models generate images natively via generateContent with responseModalities: ['IMAGE'] (or return audio/other binary data), the inlineData parts in streamed chunks are silently lost because the aggregation loop does not handle them.

Non-streaming generateContent calls are unaffected — the plugin logs the full raw response as output.

What instrumentation is missing

In js/src/instrumentation/plugins/google-genai-plugin.ts, the aggregateGenerateContentChunks function (lines 749–778) processes parts from streamed chunks:

for (const part of candidate.content.parts) {
  if (part.text !== undefined) {
    // handled ✓
  } else if (part.functionCall) {
    // handled ✓
  } else if (part.codeExecutionResult) {
    // handled ✓
  } else if (part.executableCode) {
    // handled ✓
  }
  // inlineData → falls through, silently dropped ✗
}

The vendored type GoogleGenAIPart in js/src/vendor-sdk-types/google-genai.ts already declares the inlineData field (line 53), but the aggregation code never handles it. Any inlineData part in a streamed chunk is silently excluded from the aggregated output span.

Impact

  • Native image generation via Gemini models (gemini-2.0-flash, etc.) with streaming produces spans where generated images are missing from the output
  • Braintrust docs state "Streaming responses are fully supported — Braintrust automatically collects streamed chunks and logs the complete response as a single span," but this is not the case for image/audio output
  • Users who stream generateContent calls with responseModalities: ['IMAGE', 'TEXT'] will see text in their spans but not the generated images

Braintrust docs status

unclear — Braintrust docs at https://www.braintrust.dev/docs/instrument/wrap-providers list @google/genai as supported and claim full streaming support, but do not specifically address image output in streamed responses.

Upstream reference

  • Google GenAI native image generation: https://ai.google.dev/gemini-api/docs/image-generation
  • generateContent with responseModalities: ['IMAGE'] returns inlineData parts containing generated images
  • This is a stable feature available on Gemini 2.0 Flash and later models

Local files inspected

  • js/src/instrumentation/plugins/google-genai-plugin.ts (lines 749–778: aggregateGenerateContentChunks part processing loop)
  • js/src/vendor-sdk-types/google-genai.ts (line 53: inlineData field on GoogleGenAIPart)
  • js/src/wrappers/google-genai.ts (wrapper proxies generateContentStream to channel)
  • e2e/scenarios/google-genai-instrumentation/ (no test cases with image output in streamed responses)

Note

This is distinct from #1673 (models.generateImages() not instrumented), which covers the dedicated Imagen API. This issue is about the standard generateContent/generateContentStream API producing image output that gets lost specifically in the streaming aggregation path.

主要语言
TypeScript
星标
29
派生
18
平均合并
1 天 15 小时
30 天内合并 PR
69

环境准备

  • 提供 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 没有贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

braintrustdata/braintrust-sdk-javascript 的其他 Issue

查看 braintrustdata/braintrust-sdk-javascript 的全部 Issue

相似的 Issue

更多 TypeScript Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。