Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Operator limits: exact model cost of Diff Reviews

Chiusa
  • Qualcosa non va?
#274 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
12/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python, typescript
Ambito
backend

Direzione di ricerca

Start in runner/.../pr_agent_runtime.py, where costUsd is currently summed from PR-Agent's own price table, and read the Dynamic Review accounting proxy from #267 that this must match. The first step is the design spike: choose one of the three listed mechanisms and record it in runner/README.md. Done means the cost is the exact sum of provider-reported costs, null when any call is unpriced, and tests T01 to T08 pass.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

area: runner type: feature

Part of #268. Depends on #272 and #267.

Problem

#267 makes the cost of a Dynamic Review exact, because its model calls pass through the runner's accounting proxy and OpenRouter states the charge of every request. A Diff Review does not: PR-Agent calls the model itself through LiteLLM, and the costUsd it reports (runner/.../pr_agent_runtime.py) is a sum over the calls PR-Agent could price with its own price table, which can be missing, stale or different from the charge OpenRouter actually made. For a workspace that runs on the operator's key, Diff Review spend must be measured with the same exactness as Dynamic Review, or the usage records of #272 would be exact for one kind and approximate for the other.

Behavior

  • Decision first (spike inside this issue). Pick one mechanism, record it in the issue and runner/README.md, then implement it. Candidates, in order of preference:
    1. Route PR-Agent's OpenRouter calls through the same loopback accounting proxy the Dynamic Review uses (per-job capability, fixed destination, no agent-supplied routing), reading usage.cost of every answer, with #267's rules.
    2. Read the cost from LiteLLM's response object where OpenRouter's usage.cost is passed through, if it is the provider's value and not a table lookup.
    3. Ask OpenRouter for the generation record of each request id after the run (GET /api/v1/generation?id=…), if the ids are available and the lookup is cheap.
  • Whatever is chosen: the result's usage.costUsd is the exact sum of the provider-reported costs of all calls, all or nothing as in #267 (any unpriced call makes it null), never derived from a price table.
  • It applies to every Diff job, not only managed ones (a consistent shape; own-key jobs just do not use it for limits).
  • The request path still goes to OpenRouter's fixed address with the same credential as today; provider.require_parameters (#270) is unaffected.
  • If the chosen mechanism cannot be made exact for fallback-model retries and chunked reviews, the issue says exactly what is exact and what is null, and the tests pin it.

Scope

Runner Diff path, fakes (fake LLM already returns usage; extend it with cost), tests, runner/README.md, contract text for usage.costUsd.

Out of scope

The Dynamic path (#267), how cost is used, changing the review quality or prompts.

Test cases

ID kind level case expected
T01 behavior unit (runner) a Diff run with three model calls that each report a cost usage.costUsd is their exact decimal sum
T02 behavior unit (runner) one call has no cost costUsd is null, tokens still reported
T03 behavior unit (runner) fallback model is used after the first failed both attempts' costs are included, or the rule the spike set is applied and pinned
T04 behavior unit (runner) chunked review with several calls all calls counted
T05 regression integration (runner) Diff run against the fake LLM result unchanged except for the exact cost
T06 security unit the model destination and key are not controllable from the pull request or the model as before
T07 contract unit result validation in the runner and the app accepts the cost unchanged
T08 regression unit no price table or per-token rate used for the reported cost pinned by a test
Lingua principale
TypeScript
Stelle
0
Fork
1
Merge medio
1h 15m
PR unite (30g)
217

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di ExpidetGames/code-reviewer

Tutte le issue di ExpidetGames/code-reviewer

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.