Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

@tanstack/ai-anthropic: a stream that fails after message_start loses the call's input usage

Abierto
  • ¿Algo no está bien?
#1,735 2 comentarios 0 reacciones 1 asignado Ver en GitHub

Los mantenedores suelen responder en 1 día

@AlemTuzlak ya está trabajando en esto.

Desde el 13/10/2026.

  • #1793 de @ktrz — abierto

Evaluación

Este issue todavía no se ha evaluado.

Descripción

has-pr waiting-on: maintainer

Package / version: @tanstack/ai-anthropic 0.21.1, @tanstack/ai 0.68.0, @anthropic-ai/sdk 0.97.1, Node 24.20.0. The same code is on main.

Summary

Anthropic reports a call's input side (input_tokens, cache_creation_input_tokens, cache_read_input_tokens) in the stream's first event, message_start. processAnthropicStream stores it in messageStartUsage and only uses it when the closing message_delta arrives (packages/ai-anthropic/src/adapters/text.ts, around lines 1569 and 2025 on main). If the stream fails between those two events, a connection reset, a mid-stream error event, an overloaded_error, the catch block yields RUN_ERROR with no usage, and the input tokens, which Anthropic has already counted and bills, are lost to the consumer. No chunk before RUN_FINISHED carries usage either, so there is nothing earlier to read it from.

This is the sibling of #1597, which fixed the max_tokens stop: that case has a closing message_delta to build from. The failure case has only message_start.

Reproduction

A fetch whose response body drops after the second content_block_delta. Everything up to the cut is Anthropic's real stream.

import { chat } from '@tanstack/ai'
import { createAnthropicChatWithClient } from '@tanstack/ai-anthropic'
import Anthropic from '@anthropic-ai/sdk'

const MODEL = 'claude-haiku-4-5'
const messages = [{ role: 'user', content: 'Count from one to thirty, one number per line.' }]

const cuttingFetch = async (url, init) => {
  const res = await fetch(url, init)
  let deltas = 0
  const decoder = new TextDecoder()
  const body = res.body.pipeThrough(
    new TransformStream({
      transform(chunk, controller) {
        deltas += (decoder.decode(chunk, { stream: true }).match(/event: content_block_delta/g) ?? []).length
        if (deltas > 2) controller.error(Object.assign(new Error('read ECONNRESET'), { code: 'ECONNRESET' }))
        else controller.enqueue(chunk)
      },
    }),
  )
  return new Response(body, { status: res.status, headers: res.headers })
}

const client = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY, fetch: cuttingFetch, maxRetries: 0 })
const adapter = createAnthropicChatWithClient(MODEL, client)
const seen = []
for await (const c of chat({ adapter, messages })) seen.push(c)
console.log('chunk types seen:', seen.map((c) => c.type).join(' '))
console.log('chunks carrying usage:', seen.filter((c) => c.usage !== undefined).map((c) => c.type).join(' ') || '(none)')
console.log('last chunk =', JSON.stringify(seen.at(-1)))

// What Anthropic sent in its first event for the same request.
const plain = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY })
const stream = await plain.messages.create({ model: MODEL, max_tokens: 64, messages, stream: true })
for await (const e of stream) {
  if (e.type === 'message_start') { console.log('raw message_start usage =', JSON.stringify(e.message.usage)); break }
}

Observed

chunk types seen: RUN_STARTED TEXT_MESSAGE_START TEXT_MESSAGE_CONTENT RUN_ERROR
chunks carrying usage: (none)
last chunk = {"type":"RUN_ERROR","timestamp":1791850527495,"message":"read ECONNRESET","code":"ECONNRESET","metadata":{"tanstack":{"model":"claude-haiku-4-5"}}}
raw message_start usage = {"input_tokens":18,"cache_creation_input_tokens":0,"cache_read_input_tokens":0,"cache_creation":{"ephemeral_5m_input_tokens":0,"ephemeral_1h_input_tokens":0},"output_tokens":1,"service_tier":"standard","inference_geo":"not_available"}

The adapter held input_tokens: 18 from the first event and discarded it. With a cached system prompt the discarded number is the cache read or write count for the whole prompt, which is most of the call's price.

Expected

The input side reaches the consumer when the stream fails after message_start. Two ways that could look, not exclusive:

  1. Usage on the error. In the catch of processAnthropicStream, when messageStartUsage is set, attach usage: buildAnthropicUsage(messageStartUsage) (output count zero or whatever message_start carried) to the RUN_ERROR, the way #1597 put usage on the max_tokens error. Smallest change, and it covers every mid-stream failure the adapter sees.

  2. Opening usage as an event. Yield the message_start usage when it arrives, so a consumer can record the input side before the run ends. That also covers the case the adapter cannot see: the process itself dying mid-stream. This is what I'm carrying as a local patch to the built file today:

if (event.type === 'message_start') {
  messageStartUsage = event.message.usage
  yield {
    type: EventType.CUSTOM,
    name: 'anthropic.message_start',
    value: { model: event.message.model, usage: event.message.usage },
    model,
    timestamp: Date.now(),
  }
} else if (event.type === 'content_block_start') {

A provider-named CUSTOM event is the least intrusive shape for a patch, but a cross-adapter one (usage on RUN_STARTED, or a shared early-usage event the OpenAI Responses and Bedrock adapters could also emit) would be the better API.

Do you intend to try to help solve this bug with your own PR? No, because I do not have time to dig into it. The two shapes above are what I found while patching locally; I won't be sending a PR.

  • I agree to follow this project's Code of Conduct
  • I understand that if my bug cannot be reliably reproduced in a debuggable environment, it will probably not be fixed and this issue may even be closed.
Lenguaje dominante
TypeScript
Estrellas
3.2k
Forks
365
Merge medio
1 d 9 h
PR fusionados (30 d)
242

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de TanStack/ai

Todos los issues de TanStack/ai

Issues similares

Más issues de TypeScript

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.