StreamProcessor folds long text and tool-call streams in quadratic memory and time

オープン
  • 何か問題がありますか？
#1,963 コメント 0 件 リアクション 0 件 担当者 1 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

@jherr がすでに取り組んでいます。

2026年11月1日 から。

評価

この issue はまだ評価されていません。

説明

waiting-on: maintainer
Describe the bug

StreamProcessor gets slower and heavier with every delta of one message. Two places cause it, both still in @tanstack/ai 1.0.1 (packages/ai/src/activities/chat/stream/processor.ts).

1. Text deltas hold a copy of the whole message per delta (memory and time).

handleTextMessageContentEvent, line 2343:

const nextText = delta !== '' ? currentText + delta : currentText
const textDelta = nextText.slice(currentText.length)
state.currentSegmentText = nextText
state.totalTextContent += textDelta

In V8, currentText + delta builds a rope. slice on a rope first flattens it into one buffer. For a result of 13 characters or more it then returns a sliced string that points into that buffer, so the delta is not copied out. totalTextContent concatenates those slices, and each one keeps a full flattened copy of the message alive. One message of 10,000 deltas of 16 characters (160 KB of text) holds about 760 MB after a forced GC. 5,000 deltas of 200 characters run Node out of memory at the default 4 GB heap.

textDelta always equals delta here, so the fix is:

state.currentSegmentText = nextText
state.totalTextContent += delta

2. Tool-call arguments are re-parsed in full on every delta (time).

handleToolCallArgsEvent, line 2515:

existingToolCall.arguments += chunk.delta || ''
existingToolCall.parsedArguments = this.jsonParser.parse(existingToolCall.arguments)

The partial parse is O(length of the arguments so far), so a call whose arguments arrive in n deltas costs O(n²). A tool writing a large file streams thousands of TOOL_CALL_ARGS deltas: 4,000 deltas of 100 characters take about 2 s, 10,000 about 12 s. Within the processor only getState() reads parsedArguments before TOOL_CALL_END or the completion path sets the final value. One fix is to parse on read:

// getState()
for (const [id, tc] of state.toolCalls) {
  if (tc.state === 'awaiting-input' || tc.state === 'input-streaming') {
    tc.parsedArguments = this.jsonParser.parse(tc.arguments)
  }
  toolCalls.set(id, tc)
}
Reproduction
import { StreamProcessor } from '@tanstack/ai'
import { setFlagsFromString } from 'node:v8'
import { runInNewContext } from 'node:vm'

setFlagsFromString('--expose-gc')
const gc = runInNewContext('gc')

gc()
const before = process.memoryUsage().heapUsed
const p = new StreamProcessor()
p.processChunk({ type: 'RUN_STARTED', runId: 'r', threadId: 't' })
p.processChunk({ type: 'TEXT_MESSAGE_START', messageId: 'm', role: 'assistant' })
for (let i = 0; i < 10_000; i++)
  p.processChunk({ type: 'TEXT_MESSAGE_CONTENT', messageId: 'm', delta: 'x'.repeat(16) })
gc()
const held = Math.round((process.memoryUsage().heapUsed - before) / 1e6)
console.log(held, 'MB held for', p.getMessages()[0].parts[0].content.length, 'characters')
// 0.61.0, whose code here matches 1.0.1: ~800 MB. With the fix: ~1 MB.

let parsed = 0
const q = new StreamProcessor({ jsonParser: { parse: (s) => ((parsed += s.length), undefined) } })
q.processChunk({ type: 'RUN_STARTED', runId: 'r', threadId: 't' })
q.processChunk({ type: 'TEXT_MESSAGE_START', messageId: 'm', role: 'assistant' })
q.processChunk({ type: 'TOOL_CALL_START', toolCallId: 'c', toolCallName: 'Write', parentMessageId: 'm' })
for (let i = 0; i < 10_000; i++)
  q.processChunk({ type: 'TOOL_CALL_ARGS', toolCallId: 'c', delta: 'x'.repeat(10) })
console.log(parsed, 'characters parsed for 100,000 characters of arguments')
// 0.61.0: 500,050,000. With the fix: 0.
Expected behavior

Folding a message of n deltas takes time and memory linear in its length.

Environment

@tanstack/ai 0.61.0 and 1.0.1, Node 24.21. We run the two fixes above as a local patch on 0.61.0 and can open a PR if that helps.

主要言語
TypeScript
スター
3.3k
フォーク
378
平均マージ
1日 1分
マージ済み PR（30日）
208

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

TanStack/ai のほかの issue

TanStack/ai の issue をすべて見る

似ている issue

TypeScript の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。