`ChatEngine.handleTextMessageContentEvent` is O(n²) in answer length: `startsWith(before)` copies the whole accumulated text on every chunk

已关闭
  • 哪里不对？
#1,951 0 条评论 0 个 reaction 已指派 1 人 在 GitHub 查看

维护者通常 1 天内回复

@tombeckenham 已经在做这个了。

开始于 2026年10月30日。

  • #1962 来自 @tombeckenham —— 已合并

评估

这个 Issue 还没有评估数据。

描述

has-pr waiting-on: maintainer
What happens

handleTextMessageContentEvent in packages/ai/src/activities/chat/index.ts (main @ a7eb584, L2405–L2431) runs this for every TEXT_MESSAGE_CONTENT chunk:

const before = this.accumulatedContent
if (typeof extra.content === 'string' && extra.content !== '') {
  this.accumulatedContent = extra.content
} else {
  this.accumulatedContent += chunk.delta
}
// ...
if (!this.accumulatedContent.startsWith(before)) {
  this.turnParts = null
  return
}
const delta = this.accumulatedContent.slice(before.length)

After +=, V8 doesn't copy the text. It stores the result as a string that points to the two parts. startsWith then forces V8 to copy the whole accumulated answer into one string, on every chunk. So the total cost grows with the square of the answer length, and none of it yields to the event loop.

Reproduction

This uses the same string operations, with no TanStack code involved. Node 22, Apple Silicon:

const N = 195_000
const chunk = 'abcd'

let acc = ''
const t0 = performance.now()
for (let i = 0; i < N / chunk.length; i++) {
  const before = acc
  acc += chunk
  if (!acc.startsWith(before)) {
    throw new Error('unreachable')
  }
  acc.slice(before.length)
}
console.log(performance.now() - t0)
  • With the startsWith, it takes about 10.9 s.
  • With only the +=, it takes about 0 ms.

In a real chat() run, a CPU profile of a stream of about 195k characters in 4-character chunks showed about 15.7 s inside handleStreamChunk. A model that returns a long answer in small chunks blocks the server's event loop for that whole time.

We noticed it in a test that streams a large fake answer four characters at a time. Writing the same bytes as 3-byte CJK characters cut the character count to a quarter and the runtime by more than ten times, about what O(n²) in string length predicts. A model that streams a long answer in small deltas hits the same cost in production.

Affected versions: @tanstack/ai 1.0.0 and 1.0.1, and current main.

Suggested fix

The prefix check only matters when an adapter sends a cumulative content snapshot. When the chunk is a plain delta, the new text starts with before by definition, and the new text is chunk.delta. So the check can move into the snapshot branch:

const extra = chunk as AdapterYieldChunk
const before = this.accumulatedContent
let delta: string
if (typeof extra.content === 'string' && extra.content !== '') {
  this.accumulatedContent = extra.content
  if (this.turnParts && !extra.content.startsWith(before)) {
    // A cumulative `content` snapshot rewrote earlier text; order unknown.
    this.turnParts = null
  }
  delta = extra.content.slice(before.length)
} else {
  this.accumulatedContent += chunk.delta
  delta = chunk.delta
}
this.middlewareCtx.accumulatedContent = this.accumulatedContent
if (!this.turnParts) return
if (delta === '') return
const last = this.turnParts[this.turnParts.length - 1]
if (last && last.type === 'text') {
  last.content += delta
} else {
  this.turnParts.push({
    type: 'text',
    content: delta,
  })
}

With this change, a stream of deltas does constant work per chunk. Snapshot chunks still copy the text, but they are the rare fallback path.

A regression test could stream about 200k characters of 4-character deltas through chat() with a mock adapter, then check that the turn's text part matches the input and that the run takes well under a second. If you'd rather not test timing, assert on the result and treat the benchmark above as the guard.

主要语言
TypeScript
星标
3.3k
派生
378
平均合并
1 天 2 小时
30 天内合并 PR
248

环境准备

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

TanStack/ai 的其他 Issue

查看 TanStack/ai 的全部 Issue

相似的 Issue

更多 TypeScript Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。