Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Refreshing the bundled price table silently downgrades models LiteLLM has pruned

Aperta
  • Qualcosa non va?
#1,410 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
18/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
go
Ambito
data

Direzione di ricerca

Start with the pricegen generator and its Entries function, then read TestBundled_DistinguishesOpusVersions, TestBundled_LongContextThresholdIsCarried and TestBundled_MatchesSnapshot to see what the bundled table currently guarantees. The issue first needs a maintainer decision on how long a pruned rate is trusted and where a retained rate lives, so do not start on code yet. Done would be a settled policy plus a test that catches any pruned model, not just the two named ones.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

The bundled price table is generated from a pinned LiteLLM commit. Moving that pin forward is currently not safe: upstream prunes rows, and every pruned model silently falls back to its family glob at the wrong rate.

Found while trying to bump the pin to pick up gpt-6-luna for Codex (#1365). The bump was abandoned — the trade was one OpenAI model gained against eight Anthropic models mispriced.

What happens

Regenerating at the newest upstream commit (314f1798) against the current pin (ee7c7e14):

  • 21 previously-priced models lose their exact rows, including claude-3-7-sonnet-20250219, claude-3-opus-20240229, claude-4-opus-20250514, claude-4-sonnet-20250514, claude-opus-4-1, claude-opus-4-1-20250805, claude-opus-4-20250514, claude-sonnet-4-20250514
  • claude-sonnet-4-5 loses its long-context premium — upstream now reports no input_cost_per_token_above_200k_tokens
  • 15 models are gained

With an exact row gone, the model resolves through its family glob, which carries the newest family member's rates. For claude-opus-4-1 that is $4/Mtok against its real $15/Mtok — a 3.75× under-charge.

That is precisely the error the exact rows were added to prevent. pricegen's own doc says so:

"the data already distinguishes claude-opus-4-1 ($15/Mtok input) from claude-opus-5 ($5/Mtok), which the three hand-measured family globs it replaces priced identically — a 3x error on every opus-4-1 request."

Two existing tests catch it, which is the only reason it was not shipped: TestBundled_DistinguishesOpusVersions and TestBundled_LongContextThresholdIsCarried. They are the current safety net, and they are incidental — they happen to name two affected models. A future pruning of a model neither test mentions would pass.

Why upstream does it, and why the old rate is still right

Not a bug on their side. claude-opus-4-1 still exists upstream, but only under resellers:

anthropic.claude-opus-4-1-20250805-v1:0      bedrock_converse   $15/Mtok
us.anthropic.claude-opus-4-1-20250805-v1:0   bedrock_converse   $15/Mtok
databricks/databricks-claude-opus-4-1        databricks         $15/Mtok

The first-party anthropic row was removed; the remaining first-party opus rows are 4-5 through 4-8. So LiteLLM prunes first-party rows for models Anthropic has retired from its own API, and the mirrors stay because Bedrock and Databricks still serve them.

That tells us who is affected, and it is not nobody: a request to api.anthropic.com for a retired model gets an error rather than a mispriced bill, but the bundled table is Host: "*" — it prices on model name regardless of endpoint. So a Bedrock or Databricks user running claude-opus-4-1 keeps running it, at $15/Mtok, and would be charged $4.

It also means the pruned rate is still correct for whoever can still call the model. This is not stale data; it is data upstream stopped carrying for one endpoint.

Suggested fix

Have Entries union the new snapshot with the exact rows the table already carried, so an upstream pruning cannot downgrade a model to its family glob. "Never forget a rate we once knew."

The policy question that needs deciding first, and the reason this is an issue rather than a PR:

  • How long is a retired rate trusted? It is right until the vendor reprices a model they no longer sell on that endpoint — unlikely, but unbounded. Options: keep forever; keep with the commit it was last seen in, so the table records its own staleness; or expire after N refreshes.
  • Where does a kept row live? Carrying it in the generated file needs the generator to read its own previous output, which is a new input to a transform whose reproducibility is currently its selling point (TestBundled_MatchesSnapshot re-runs it over the snapshot and compares). A separate checked-in "retained rates" file keeps the transform pure.
  • Should provenance distinguish them? A rate upstream no longer publishes is arguably weaker than one it does, and Provenance already exists to express that.

Meanwhile

  • The pin stays where it is; the suite is green.
  • Codex is priced by a pricing: entry instead — documented in docs/agents/codex.md, verified end to end ($0.000335 on a real turn).
  • Anyone needing one of the 21 pruned models priced can do the same, and a configured entry always beats the bundled table.

Also worth knowing

The reverse hazard exists too: a bump also gains 15 models, and any that are dearer than ~$100/Mtok are dropped by pricing.BundleableRate — see #1406. So a refresh can change coverage in both directions, which strengthens the case for the diff being inspectable rather than trusted.

Refs #1365, #1406, #942.

Assisted-By: Claude (Anthropic AI) [email protected]

Lingua principale
Go
Stelle
16
Fork
41
Merge medio
11h 23m
PR unite (30g)
275

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di rossoctl/cortex

Tutte le issue di rossoctl/cortex

Issue simili

Altre issue su Go

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.