Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Refreshing the bundled price table silently downgrades models LiteLLM has pruned

Abierto
  • ¿Algo no está bien?
#1,410 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
18/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
go
Área
data

Línea de trabajo

Start with the pricegen generator and its Entries function, then read TestBundled_DistinguishesOpusVersions, TestBundled_LongContextThresholdIsCarried and TestBundled_MatchesSnapshot to see what the bundled table currently guarantees. The issue first needs a maintainer decision on how long a pruned rate is trusted and where a retained rate lives, so do not start on code yet. Done would be a settled policy plus a test that catches any pruned model, not just the two named ones.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

The bundled price table is generated from a pinned LiteLLM commit. Moving that pin forward is currently not safe: upstream prunes rows, and every pruned model silently falls back to its family glob at the wrong rate.

Found while trying to bump the pin to pick up gpt-6-luna for Codex (#1365). The bump was abandoned — the trade was one OpenAI model gained against eight Anthropic models mispriced.

What happens

Regenerating at the newest upstream commit (314f1798) against the current pin (ee7c7e14):

  • 21 previously-priced models lose their exact rows, including claude-3-7-sonnet-20250219, claude-3-opus-20240229, claude-4-opus-20250514, claude-4-sonnet-20250514, claude-opus-4-1, claude-opus-4-1-20250805, claude-opus-4-20250514, claude-sonnet-4-20250514
  • claude-sonnet-4-5 loses its long-context premium — upstream now reports no input_cost_per_token_above_200k_tokens
  • 15 models are gained

With an exact row gone, the model resolves through its family glob, which carries the newest family member's rates. For claude-opus-4-1 that is $4/Mtok against its real $15/Mtok — a 3.75× under-charge.

That is precisely the error the exact rows were added to prevent. pricegen's own doc says so:

"the data already distinguishes claude-opus-4-1 ($15/Mtok input) from claude-opus-5 ($5/Mtok), which the three hand-measured family globs it replaces priced identically — a 3x error on every opus-4-1 request."

Two existing tests catch it, which is the only reason it was not shipped: TestBundled_DistinguishesOpusVersions and TestBundled_LongContextThresholdIsCarried. They are the current safety net, and they are incidental — they happen to name two affected models. A future pruning of a model neither test mentions would pass.

Why upstream does it, and why the old rate is still right

Not a bug on their side. claude-opus-4-1 still exists upstream, but only under resellers:

anthropic.claude-opus-4-1-20250805-v1:0      bedrock_converse   $15/Mtok
us.anthropic.claude-opus-4-1-20250805-v1:0   bedrock_converse   $15/Mtok
databricks/databricks-claude-opus-4-1        databricks         $15/Mtok

The first-party anthropic row was removed; the remaining first-party opus rows are 4-5 through 4-8. So LiteLLM prunes first-party rows for models Anthropic has retired from its own API, and the mirrors stay because Bedrock and Databricks still serve them.

That tells us who is affected, and it is not nobody: a request to api.anthropic.com for a retired model gets an error rather than a mispriced bill, but the bundled table is Host: "*" — it prices on model name regardless of endpoint. So a Bedrock or Databricks user running claude-opus-4-1 keeps running it, at $15/Mtok, and would be charged $4.

It also means the pruned rate is still correct for whoever can still call the model. This is not stale data; it is data upstream stopped carrying for one endpoint.

Suggested fix

Have Entries union the new snapshot with the exact rows the table already carried, so an upstream pruning cannot downgrade a model to its family glob. "Never forget a rate we once knew."

The policy question that needs deciding first, and the reason this is an issue rather than a PR:

  • How long is a retired rate trusted? It is right until the vendor reprices a model they no longer sell on that endpoint — unlikely, but unbounded. Options: keep forever; keep with the commit it was last seen in, so the table records its own staleness; or expire after N refreshes.
  • Where does a kept row live? Carrying it in the generated file needs the generator to read its own previous output, which is a new input to a transform whose reproducibility is currently its selling point (TestBundled_MatchesSnapshot re-runs it over the snapshot and compares). A separate checked-in "retained rates" file keeps the transform pure.
  • Should provenance distinguish them? A rate upstream no longer publishes is arguably weaker than one it does, and Provenance already exists to express that.

Meanwhile

  • The pin stays where it is; the suite is green.
  • Codex is priced by a pricing: entry instead — documented in docs/agents/codex.md, verified end to end ($0.000335 on a real turn).
  • Anyone needing one of the 21 pruned models priced can do the same, and a configured entry always beats the bundled table.

Also worth knowing

The reverse hazard exists too: a bump also gains 15 models, and any that are dearer than ~$100/Mtok are dropped by pricing.BundleableRate — see #1406. So a refresh can change coverage in both directions, which strengthens the case for the diff being inspectable rather than trusted.

Refs #1365, #1406, #942.

Assisted-By: Claude (Anthropic AI) [email protected]

Lenguaje dominante
Go
Estrellas
18
Forks
42
Merge medio
12 h 20 min
PR fusionados (30 d)
229

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de rossoctl/cortex

Todos los issues de rossoctl/cortex

Issues similares

Más issues de Go

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.