[bot] Vercel AI SDK: `experimental_evaluate` instrumentation is stale after upstream rename to `experimental_decide`
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
- 难度
- 3/5
- 预计耗时
- 1-2 天
- 新手友好度
- 45/100
- Issue 类型
- 功能
- 描述清晰度
- 基本清楚
- 活跃度
- 活跃
- 技术栈
- javascript, typescript
- 领域
- ai, observability
调研方向
Start with the sole Vercel AI SDK entry in js/src/auto-instrumentations/configs/ai-sdk.ts (lines 18-31), which targets functionQuery experimental_evaluate for the ai >=7.0.103 <8.0.0 range, and read the comment above it. Check how the existing config entries and tests in that directory are structured, then see whether a decision-feature entry for experimental_decide fits the same pattern. Done when a call through experimental_decide yields a Braintrust span and the config is covered by a test.
由索引模型根据 Issue 内容生成。
描述
Summary
This repo's Vercel AI SDK instrumentation targets the function export experimental_evaluate to trace the AI SDK's structured-decision feature, but upstream renamed that export to experimental_decide in [email protected] ("feat: rename evaluate to decide"). The existing instrumentation config's version range (>=7.0.103 <8.0.0) still nominally spans the current ai release line (latest: 7.0.140), but it is matching against an export name (experimental_evaluate) that no longer corresponds to the feature's current public entrypoint upstream. This is an existing-instrumentation-goes-stale-after-upstream-rename gap, not a never-instrumented surface: calls made through the current public function name produce no Braintrust span.
What instrumentation is missing
experimental_decide(...)— the current public export (post-rename) for the AI SDK's structured decision/evaluation feature. Upstream continued developing this feature under the new name after the rename, including: image input support for "OpenAI Decisions and language model adapters" ([email protected]), ordered text/file/JSON parts in decision state ([email protected]), and serialization of decision file bytes as base64 in OpenTelemetry attributes — i.e. upstream already emits its own OTel attributes for this feature that this repo's tracing does not pick up.
Current (stale) coverage:
js/src/auto-instrumentations/configs/ai-sdk.ts:18-31— the sole config entry for this feature targetsfunctionQuery: { functionName: "experimental_evaluate", isExportAlias: true }againstai>=7.0.103 <8.0.0,dist/index.js. No entry anywhere in this file (orjs/src/instrumentation/plugins/ai-sdk-*) referencesexperimental_decide.- A repo-wide case-insensitive search for
experimental_decide/generateDecisionunderjs/src/returns no matches.
Braintrust docs status
not_found — https://www.braintrust.dev/docs/integrations/ai-providers/vercel-ai-sdk and the Vercel AI SDK tracing cookbook describe Braintrust's AI SDK span coverage (generateText, streamText, generateObject, streamObject, tool-call child spans) and do not mention experimental_evaluate, experimental_decide, or a decision/evaluation span anywhere.
Upstream references
aipackage changelog (raw): https://raw.githubusercontent.com/vercel/ai/main/packages/ai/CHANGELOG.md7.0.128: "feat: renameevaluatetodecide"7.0.133: "Add ordered text, file, and JSON parts to experimental decision state"; image input support for "OpenAI Decisions and language model adapters"; "Serialize decision file bytes as base64" in OpenTelemetry attributes7.0.139: "Fix validation of rounded choice probabilities inexperimental_decide"- Latest version in changelog:
7.0.140
Local files inspected
js/src/auto-instrumentations/configs/ai-sdk.ts— full file read; onlyexperimental_evaluateis targeted, with a comment noting "Evaluation does not emit v7 telemetry callbacks. Instrument its public export."- Repo-wide case-insensitive grep for
experimental_decide,generateDecision,decisionsunderjs/src/— no relevant matches
- 主要语言
- TypeScript
- 星标
- 29
- 派生
- 16
- 平均合并
- 1 天 19 小时
- 30 天内合并 PR
- 69
环境准备
- 提供 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 没有贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
braintrustdata/braintrust-sdk-javascript 的其他 Issue
-
难度 4/5 3-5 天 新手友好度 45/100
braintrustdata/braintrust-sdk-javascript#2618 ·
维护者通常 1 天内回复
-
typescript
难度 4/5 3-5 天 新手友好度 48/100
braintrustdata/braintrust-sdk-javascript#2605 ·
维护者通常 1 天内回复
-
typescript
难度 4/5 3-5 天 新手友好度 52/100
braintrustdata/braintrust-sdk-javascript#2603 ·
维护者通常 1 天内回复
-
typescript
难度 4/5 3-5 天 新手友好度 55/100
braintrustdata/braintrust-sdk-javascript#2578 ·
维护者通常 1 天内回复
-
[bot] Instrument Letta Agent SDK (143,425 weekly downloads)可能已有人在做 @karthiksenv 于 1 天前认领。 未关闭typescript
难度 4/5 3-5 天 新手友好度 55/100
braintrustdata/braintrust-sdk-javascript#2577 ·
维护者通常 1 天内回复
查看 braintrustdata/braintrust-sdk-javascript 的全部 Issue
相似的 Issue
-
Create/import wallet form shows raw "password must be at least 1 characters" instead of the translated message可能已有人在做 @CapThunder19 今天认领。 未关闭bug
难度 2/5 1-3 小时 新手友好度 82/100
joinmarket-webui/jam#1538 ·
维护者通常 1 天内回复
-
t0ggles
难度 2/5 1-3 小时 新手友好度 70/100
nolimits4web/swiper#8240 · 1 条评论 ·
-
难度 2/5 1-3 小时 新手友好度 82/100
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 72/100
modelcontextprotocol/inspector#2723 ·
维护者通常 1 天内回复
-
status:approved type:bug
难度 1/5 1 小时以内 新手友好度 70/100
维护者通常 1 天内回复