Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

AI SDK tool spans don't respect `toModelOutput`

Closed
  • Is something wrong?
#2,651 1 comment 0 reactions 1 assignee View on GitHub

Maintainers usually reply within 1 day

@AbhiPrasad is already working on this.

Since Oct 22, 2026.

  • #2652 by @AbhiPrasad — merged

Assessment

This issue has not been assessed yet.

Description

typescript

When a Vercel AI SDK tool defines toModelOutput, the model only sees that transformed value, but the wrapAISDK tool span still logs the raw execute result via the patched execute.

Apps use toModelOutput to keep data away from the model, for example query rows a user hasn't agreed to share. Braintrust's global setMaskingFunction doesn't fix this cleanly, because it doesn't know which tool a value came from.

Repro with [email protected] and [email protected], using a mock model:

import * as ai from 'ai'
import { MockLanguageModelV4 } from 'ai/test'
import { initLogger, wrapAISDK } from 'braintrust'
import { z } from 'zod'

initLogger({ projectName: 'repro' })
const { generateText } = wrapAISDK(ai)

await generateText({
  model: new MockLanguageModelV4({
    doGenerate: async () => ({
      content: [{ type: 'tool-call', toolCallId: 'call-1', toolName: 'runQuery', input: '{}' }],
      finishReason: { unified: 'tool-calls', raw: undefined },
      usage: { inputTokens: { total: 1 }, outputTokens: { total: 1 } },
      warnings: [],
    }),
  }),
  prompt: 'Count signups',
  tools: {
    runQuery: ai.tool({
      inputSchema: z.object({}),
      execute: async () => ({ rows: [{ email: '[email protected]' }] }),
      toModelOutput: () => ({ type: 'text', value: 'Rows hidden from the model' }),
    }),
  },
})

The runQuery tool span's output is { rows: [{ email: '[email protected]' }] }. I expected Rows hidden from the model. The raw rows also appear in the parent generateText span, under steps[].content[] in the tool result. The same happens with ai.registerTelemetry(braintrustAISDKTelemetry()).

tool span parent span
Image Image

When a tool defines toModelOutput, I expect the tool span and the parent span's tool results to log its result instead of the raw execute value. That way the trace matches what the model saw and avoids privacy footguns. If this is too big a change for the default, a wrapAISDK option to turn it on would cover our case.

Dominant language
TypeScript
Stars
29
Forks
21
Avg merge
1d 12h
Merged PRs (30d)
70

Getting set up

  • Ships a Dockerfile or Docker Compose file
  • No pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from braintrustdata/braintrust-sdk-javascript

All issues in braintrustdata/braintrust-sdk-javascript

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.