Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Operator limits: record usage per run with its measured cost

Closed
  • Is something wrong?
#272 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@Schacki-Jr is already working on this.

Since Oct 14, 2026.

  • #297 by @Schacki-Jr — open

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
18/100
Issue type
Feature
Clarity
Clearly specified
Activity status
Stale
Tech stack
postgresql, sqlite, typescript
Domain
backend, database

Research direction

Start in the job claim and complete paths in src/jobs, where the reserve and settle inserts go, and read src/jobs/README.md and src/runner-api/README.md. Add the usage_records table with its migration and a scoped repository, then cover it in scoping.test.ts and scoped-isolation.integration.test.ts. Done when T01 to T10 pass on both SQLite and Postgres. A linked PR (#297) is already open and the issue depends on #267, so check with maintainers before starting.

Written by the indexing model from the issue text.

Description

area: app type: feature

Part of #268. Depends on #267 (the exact model cost of Dynamic Review runs): this issue stores the cost that issue reports and does not measure anything itself.

Problem

A job's usage (tokens, costUsd) is stored inside the JSON result of the job. That is enough to show one run, but not to ask "how many Diff and Dynamic reviews did this workspace run since X" or "how much did the operator's key spend", atomically with the decision to start a run, and in a form an external service can read incrementally. Limits (#277), the usage page (#282) and the operator usage API (#278) all need that.

Behavior

  • New table usage_records, one row per job (unique job_id), scoped to the workspace: id (monotonic), workspace_id, job_id, review_kind (diff/dynamic), key_mode (own when the job ran on the workspace's key, managed when it ran on the operator's key, i.e. the source of the resolved LLM access), model_tier (nullable, filled by #280), state (reserved, settled, void), cost_micro_usd (nullable integer: the job's usage.costUsd times one million, rounded half up; null when unknown), prompt_tokens, completion_tokens (nullable), reserved_at, settled_at (null until settled).
  • Reserve. When the claim hands a job to a runner, the record is inserted in the same transaction as the claim (state reserved). A job that is claimed again after a lost lease reuses its record; it never counts twice.
  • Settle. When the job completes, the record is settled from the result: with a reported usage it is settled with its cost (possibly null) and tokens; a result with no model call made (no usage and the job failed before the model was used: checkout, configuration, lease loss that ends in a failure, cancellation) turns the record void. A record is immutable once settled or void.
  • void and reserved records exist for the counting rules of #277: reserved and settled count, void does not.
  • Reading is by repository functions only: count of records by kind since a time, sum of settled managed cost since a time, a page of settled records ordered by (settled_at, id) after a cursor.
  • Records are small and kept; no retention job in this issue.
  • A run on a workspace without any entitlement is recorded as well (key mode from the access source), so that usage exists the moment limits are switched on. This does not change what the run does.

Scope

Table, migration, repository (scoped, classified in scoping.test.ts, rows in scoped-isolation.integration.test.ts), the insert in the claim transaction, the settle in the complete path, tests, docs in src/jobs/README.md and src/runner-api/README.md.

Out of scope

Limits and refusal (#277), runs that fail after using the model (#273), the Diff cost (#274), any endpoint (#278), UI (#282).

Depends on

#267. Can run in parallel with #271.

Test cases

ID kind level case expected
T01 behavior integration a job is claimed one reserved record in the same transaction, with kind and key mode of the job
T02 behavior integration the job completes with usage.costUsd = 0.0123456 record settled, cost_micro_usd = 12346, tokens stored
T03 behavior integration the result has costUsd: null settled with null cost, not void
T04 behavior integration the job fails before any model call record void
T05 behavior integration the lease is lost and the job is claimed again the same record, still one
T06 security integration repository of workspace A cannot see or settle B's records (scoping tests)
T07 behavior integration complete is repeated (alreadyCompleted) the record is not changed again
T08 regression integration a workspace without entitlements runs exactly as before and still gets a record
T09 behavior integration page of settled records after a cursor ordered by (settled_at, id), stable under inserts, never contains reserved or void
T10 regression integration SQLite and Postgres same shape (parity test)
Dominant language
TypeScript
Stars
0
Forks
1
Avg merge
1h 20m
Merged PRs (30d)
309

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ExpidetGames/code-reviewer

All issues in ExpidetGames/code-reviewer

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.