[bot] Instrument Baseten Performance Client (87,159 weekly downloads)

Aperta
  • Qualcosa non va?
#1,038 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
25/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python
Ambito
observability

Direzione di ricerca

Start with the existing integrations under py/src/braintrust/integrations/ and the matrix in py/pyproject.toml to see how a wrapper is registered, then check py/noxfile.py for the session pattern. The issue lists five sync/async PerformanceClient entrypoints that need spans, so decide which ones to cover first. Done means a traced test session that runs against the package, but the scope and approach need maintainer input first.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

python

Summary

baseten-performance-client (import name baseten_performance_client) is Baseten's dedicated high-throughput Python client for embeddings, reranking, and classification inference against Baseten-hosted models. It is a Rust-backed (pyo3/reqwest/tokio) client distinct from both the general-purpose baseten management SDK and from calling Baseten endpoints through an OpenAI-compatible interface. This repository has zero instrumentation for it — no integration directory, wrapper, nox session, or matrix entry, and not even an incidental mention anywhere under py/.

This is a distinct execution surface from Braintrust's existing Baseten support: Braintrust currently only documents Baseten as a gateway-routed model provider for chat completions (see docs status below) — it does not cover this package's embeddings/reranking/classification execution API at all. It is also distinct from already-instrumented embedding/rerank libraries (cohere rerank, voyageai — itself still an open gap in #482) since it talks to Baseten-hosted deployments specifically via its own PerformanceClient class.

What is missing

The PerformanceClient class exposes paired sync/async execution entrypoints, none of which produce a Braintrust span today:

  • PerformanceClient.embed() / async_embed() — embedding execution against Baseten Embedding Inference deployments
  • PerformanceClient.rerank() / async_rerank() — document reranking execution
  • PerformanceClient.classify() / async_classify() — classification execution
  • PerformanceClient.batch_system_one() / async_batch_system_one() — batched "System One" decision execution
  • PerformanceClient.batch_post() / async_batch_post() — generic batched JSON request execution (e.g. against Engine-Builder-LLM completion endpoints)

All of these are high-concurrency batch execution calls (default max_concurrent_requests=256, configurable retries/hedging/timeouts via RequestProcessingPreference) that are the primary way applications drive bulk embedding/rerank/classification workloads through Baseten — currently entirely invisible to Braintrust tracing.

Weekly downloads

Weekly downloads: 87,159 (fetched 2026-11-02; https://pypistats.org/api/packages/baseten_performance_client/recent)

Braintrust docs status

not_found — checked:

Upstream sources

Local repo files inspected

  • py/src/braintrust/integrations/ — no baseten/ directory, no file matching baseten* anywhere
  • py/src/braintrust/wrappers/ — no Baseten wrapper
  • py/pyproject.toml [tool.braintrust.matrix] — no baseten entry
  • py/noxfile.py — no test_baseten session
  • Repo-wide case-insensitive grep for baseten under py/ — zero matches of any kind (not even incidental)
Lingua principale
Python
Stelle
22
Fork
30
Merge medio
9h
PR unite (30g)
128

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di braintrustdata/braintrust-sdk-python

Tutte le issue di braintrustdata/braintrust-sdk-python

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.