Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

[bot] Vercel AI SDK: `experimental_evaluate` instrumentation is stale after upstream rename to `experimental_decide`

Cerrado
  • ¿Algo no está bien?
#2,619 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

@lforst ya está trabajando en esto.

Desde el 14/10/2026.

  • #2621 de @lforst — fusionado

Evaluación

Dificultad
3/5
Tiempo estimado
1-2 días
Aptitud para principiantes
45/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
javascript, typescript
Área
ai, observability

Línea de trabajo

Start with the sole Vercel AI SDK entry in js/src/auto-instrumentations/configs/ai-sdk.ts (lines 18-31), which targets functionQuery experimental_evaluate for the ai >=7.0.103 <8.0.0 range, and read the comment above it. Check how the existing config entries and tests in that directory are structured, then see whether a decision-feature entry for experimental_decide fits the same pattern. Done when a call through experimental_decide yields a Braintrust span and the config is covered by a test.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Summary

This repo's Vercel AI SDK instrumentation targets the function export experimental_evaluate to trace the AI SDK's structured-decision feature, but upstream renamed that export to experimental_decide in [email protected] ("feat: rename evaluate to decide"). The existing instrumentation config's version range (>=7.0.103 <8.0.0) still nominally spans the current ai release line (latest: 7.0.140), but it is matching against an export name (experimental_evaluate) that no longer corresponds to the feature's current public entrypoint upstream. This is an existing-instrumentation-goes-stale-after-upstream-rename gap, not a never-instrumented surface: calls made through the current public function name produce no Braintrust span.

What instrumentation is missing

  • experimental_decide(...) — the current public export (post-rename) for the AI SDK's structured decision/evaluation feature. Upstream continued developing this feature under the new name after the rename, including: image input support for "OpenAI Decisions and language model adapters" ([email protected]), ordered text/file/JSON parts in decision state ([email protected]), and serialization of decision file bytes as base64 in OpenTelemetry attributes — i.e. upstream already emits its own OTel attributes for this feature that this repo's tracing does not pick up.

Current (stale) coverage:

  • js/src/auto-instrumentations/configs/ai-sdk.ts:18-31 — the sole config entry for this feature targets functionQuery: { functionName: "experimental_evaluate", isExportAlias: true } against ai >=7.0.103 <8.0.0, dist/index.js. No entry anywhere in this file (or js/src/instrumentation/plugins/ai-sdk-*) references experimental_decide.
  • A repo-wide case-insensitive search for experimental_decide / generateDecision under js/src/ returns no matches.

Braintrust docs status

not_found — https://www.braintrust.dev/docs/integrations/ai-providers/vercel-ai-sdk and the Vercel AI SDK tracing cookbook describe Braintrust's AI SDK span coverage (generateText, streamText, generateObject, streamObject, tool-call child spans) and do not mention experimental_evaluate, experimental_decide, or a decision/evaluation span anywhere.

Upstream references

  • ai package changelog (raw): https://raw.githubusercontent.com/vercel/ai/main/packages/ai/CHANGELOG.md
    • 7.0.128: "feat: rename evaluate to decide"
    • 7.0.133: "Add ordered text, file, and JSON parts to experimental decision state"; image input support for "OpenAI Decisions and language model adapters"; "Serialize decision file bytes as base64" in OpenTelemetry attributes
    • 7.0.139: "Fix validation of rounded choice probabilities in experimental_decide"
    • Latest version in changelog: 7.0.140

Local files inspected

  • js/src/auto-instrumentations/configs/ai-sdk.ts — full file read; only experimental_evaluate is targeted, with a comment noting "Evaluation does not emit v7 telemetry callbacks. Instrument its public export."
  • Repo-wide case-insensitive grep for experimental_decide, generateDecision, decisions under js/src/ — no relevant matches
Lenguaje dominante
TypeScript
Estrellas
29
Forks
16
Merge medio
1 d 20 h
PR fusionados (30 d)
70

Preparar el entorno

  • Incluye un Dockerfile o un archivo de Docker Compose
  • Sin plantilla de pull request
  • Sin guía de contribución

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de braintrustdata/braintrust-sdk-javascript

Todos los issues de braintrustdata/braintrust-sdk-javascript

Issues similares

Más issues de TypeScript

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.