[bot] LiteLLM: Batches API passthrough (`create_batch`/`retrieve_batch`/`list_batches`/`cancel_batch`) not instrumented

オープン
  • 何か問題がありますか？
#1,063 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
38/100
issue の種類
機能追加
明瞭さ
おおむね明確
活発さ
活発
技術スタック
python
領域
observability

調査の方向性

Start with py/src/braintrust/integrations/litellm/patchers.py, where each existing LiteLLM entry point (for example the embedding patcher) is registered in _ALL_LITELLM_PATCHERS, then read the matching wrapper in tracing.py. The batch functions need a span shape for asynchronous jobs that span several calls, so decide that design with maintainers first. Done means create, retrieve, list and cancel (sync and async) each produce a span, with tests built on the existing cassette setup.

索引モデルが issue の本文から書いたものです。

説明

python

Summary

LiteLLM ships a documented Batches API passthrough surface — litellm.create_batch / litellm.acreate_batch, litellm.retrieve_batch / litellm.aretrieve_batch, litellm.list_batches / litellm.alist_batches, and litellm.cancel_batch / litellm.acancel_batch — which proxies real async batch-inference jobs to the underlying provider's Batch API (OpenAI, Azure OpenAI, Vertex AI, Bedrock, Mistral, vLLM, xAI). This is a distinct execution surface from litellm.batch_completion (client-side fan-out over concurrent completion() calls, tracked separately and closed not_planned in #755 because it already produces spans via the patched completion function).

The Braintrust LiteLLM integration (py/src/braintrust/integrations/litellm/) instruments 22 patchers across 11 top-level entry points (completion/acompletion, text_completion/atext_completion, responses/aresponses, decisions/adecisions (+ systemone/asystemone aliases), image_generation/aimage_generation, embedding/aembedding, moderation/amoderation, speech/aspeech, transcription/atranscription, rerank/arerank), but has no patcher for any of the 8 Batches API functions. Calls made through create_batch/retrieve_batch/list_batches/cancel_batch (or their async variants) produce zero Braintrust spans — no batch submission metadata, no status, no linkage to the resulting generations.

What is missing

LiteLLM function Description Instrumented?
litellm.completion / .acompletion Single-call chat completion Yes
litellm.create_batch / .acreate_batch Submit an async batch inference job to the underlying provider's Batch API No
litellm.retrieve_batch / .aretrieve_batch Poll batch job status/result metadata No
litellm.list_batches / .alist_batches List batch jobs No
litellm.cancel_batch / .acancel_batch Cancel a batch job No

Braintrust docs status

not_found. The Braintrust LiteLLM integration page (https://www.braintrust.dev/docs/integrations/sdk-integrations/litellm) explicitly enumerates the entry points it patches — completion, acompletion, text_completion, atext_completion, responses, aresponses, image_generation, aimage_generation, embedding, aembedding, moderation, amoderation, speech, aspeech, transcription, atranscription, rerank, arerank — with no mention of create_batch/retrieve_batch/list_batches/cancel_batch anywhere on the page.

Upstream sources

  • LiteLLM Batches API docs: https://docs.litellm.ai/docs/batches — documents create_batch/acreate_batch, retrieve_batch/aretrieve_batch, list_batches/alist_batches, cancel_batch/acancel_batch as current, actively maintained functions proxying to OpenAI, Azure, Vertex AI, Bedrock, Mistral, vLLM, and xAI batch endpoints.
  • Confirmed actively maintained via recent upstream PRs, e.g. fix(vertex_ai): honor timeout on batch create, retrieve, list and cancel (BerriAI/litellm#47322) and fix(batches): register ownership for every batch create path (BerriAI/litellm#39810).

Local repo files inspected

  • py/src/braintrust/integrations/litellm/patchers.py — defines 22 patchers (LiteLLMCompletionPatcher through LiteLLMArerankPatcher); no batch patcher exists.
  • py/src/braintrust/integrations/litellm/tracing.py — wrapper implementations for all currently-patched entry points; no batch wrapper functions.
  • py/src/braintrust/integrations/litellm/integration.py — LiteLLMIntegration.patchers is sourced entirely from _ALL_LITELLM_PATCHERS in patchers.py.
  • py/src/braintrust/wrappers/litellm.py — grep for "batch" (case-insensitive): zero matches.
  • Grep for "batch" (case-insensitive) across py/src/braintrust/integrations/litellm/: only one incidental match inside a cassette YAML (unrelated content string); no code or test references to create_batch/retrieve_batch/list_batches/cancel_batch.

Relationship to existing issues

  • #755 (closed, not_planned) covers litellm.batch_completion/batch_completion_models specifically — a client-side concurrent-fan-out helper that already produces spans via the patched completion() function. This issue is about the separate, real async Batches API passthrough (create_batch et al.), which has no instrumentation path today since it doesn't call completion() internally.
主要言語
Python
スター
22
フォーク
30
平均マージ
9時間
マージ済み PR（30日）
128

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

braintrustdata/braintrust-sdk-python のほかの issue

braintrustdata/braintrust-sdk-python の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。