Hacktoberfest 2026：メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

decide() drops billed usage when a call fails after the provider responded

オープン
  • 何か問題がありますか？
#1,741 コメント 0 件 リアクション 0 件 担当者 1 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

@tombeckenham がすでに取り組んでいます。

2026年10月13日 から。

  • #1742 @ktrz による — オープン

評価

この issue はまだ評価されていません。

説明

has-pr waiting-on: maintainer
TanStack AI version

@tanstack/ai 0.68.0, @tanstack/ai-openai 0.29.1 (current main)

Framework/Library version

None. decide() runs on the server, Node.js v24.14.1.

Describe the bug and the steps to reproduce it

decide() only reports token usage on the success path. When the provider answers and bills the call, but decide() then throws, onUsage never runs and no evaluate:usage event fires. The tokens are billed but never metered. We use decide() in a planner and meter tokens through onUsage, so every one of these failures under-reports our spend.

Two concrete cases on main.

Case 1: the answers cannot be mapped. Shipped adapters pass provider answers through unchecked (for example packages/ai-typesafe/src/adapters/evaluate.ts returns json.answers as is). If an answer is missing, or has the wrong type for its question, mapAnswers throws inside decide() after adapter.evaluate() resolved. result.usage is in scope at that point and is dropped. onError runs, onUsage does not.

Case 2: OpenAI refuses a question. packages/ai-openai/src/adapters/evaluate.ts throws a plain Error('OpenAI declined to answer question "<name>"') from inside the decision.answers.map(...). The response is already in hand, so three things are lost at once: the usage from decision.usage, every other answer in the response, and any typed way to detect a refusal. The caller gets a plain Error with no code, and the adapter log entry has code: undefined. I have no deterministic repro for this one. It needs a real refusal from gpt-6-luna. The code path is visible in the file.

Maintainers already treat lost usage as a bug on the chat side: #1609 (merged) keeps Anthropic usage when a stream stops at max_tokens, and #1735 (open) reports Anthropic usage lost on a stream failure. decide() has the same hole.

Steps to reproduce case 1 on clean main:

  1. Save the test below as packages/ai/tests/decide-usage-repro.test.ts.
  2. Run pnpm nx run @tanstack/ai:test:lib --skip-nx-cache -- tests/decide-usage-repro.test.ts.
  3. The test fails with AssertionError: expected [] to deeply equal [ { promptTokens: 40, …(2) } ]. decide() threw as expected, but hookUsage and eventUsage are both empty.
import { expect, it } from 'vitest'
import { aiEventClient } from '@tanstack/ai-event-client'
import { boolean, decide } from '../src/index'
import type { TokenUsage } from '../src/types'

it('decide() reports billed usage when the answers cannot be mapped', async () => {
  const usage = { promptTokens: 40, completionTokens: 5, totalTokens: 45 }
  const hookUsage: Array<TokenUsage> = []
  const eventUsage: Array<TokenUsage> = []
  const off = aiEventClient.on('evaluate:usage', (e) => eventUsage.push(e.payload.usage), { withEventTarget: true })

  const call = decide({
    adapter: {
      kind: 'evaluate', name: 'repro', model: 'repro-model', '~types': { providerOptions: {} },
      // The provider answered and billed tokens, but the answer has the wrong type.
      evaluate: async () => ({
        model: 'repro-model', usage,
        answers: { refund: { type: 'choice', choice: 'x', probabilities: { x: 1 }, confidence: 1 } },
      }),
    },
    state: 'Please refund the extra payment.',
    questions: { refund: boolean({ instructions: 'Is this a refund request?' }) },
    middleware: [{ onUsage: (_ctx, u) => void hookUsage.push(u) }],
    debug: false,
  })

  await expect(call).rejects.toThrow('expected noul answer for "refund"')
  off()
  expect(hookUsage).toEqual([usage])
  expect(eventUsage).toEqual([usage])
})

Expected: onUsage and evaluate:usage get the usage the provider reported, then onError runs and decide() rethrows. onFinish stays off for a failed call.

Actual: nothing reports the usage.

Your Minimal, Reproducible Example - (Sandbox Highly Recommended)

Inline above. It is a Vitest test against this repo, no sandbox needed.

Screenshots or Videos (Optional)

No response

Do you intend to try to help solve this bug with your own PR?

Yes, I am also opening a PR that solves the problem along side this issue

Terms & Code of Conduct
  • I agree to follow this project's Code of Conduct
  • I understand that if my bug cannot be reliable reproduced in a debuggable environment, it will probably not be fixed and this issue may even be closed.
主要言語
TypeScript
スター
3.2k
フォーク
361
平均マージ
1日 13時間
マージ済み PR（30日）
241

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

TanStack/ai のほかの issue

TanStack/ai の issue をすべて見る

似ている issue

TypeScript の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。