Codex cost scan never completes when the 256 MiB per-file cap falls inside a multi-MB `compacted` line; Codex model breakdown unavailable
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 42/100
Research direction
Start by tracing the Codex cost scanner’s per-file byte limit, resume state handling, and SQLite persistence; the issue does not name source files or tests. Reproduce the scan with a large compacted record crossing the 256 MiB boundary, then check whether later refreshes advance the saved offset and complete the file. Done means the scan progresses or the report provides partial data with a coverage warning.
Written by the indexing model from the issue text.
Description
Environment
- CodexBar 0.84.0 (177), installed via Homebrew cask
- Codex CLI 0.170.0
- macOS 27.0.1, Apple Silicon
- Local token cost usage enabled, refresh every 5 minutes
Summary
One active Codex rollout is about 772 MB (809,825,875 bytes). The bounded scanner reads exactly maxFileBytes (268,435,456) of it and then never makes progress. While this file stays incomplete, every Codex cost refresh fails, so the Codex model and cost breakdown never appears. Claude and Antigravity cost data load normally.
Observed behavior
The app logs this about every 5 minutes, for at least 6 hours in a row:
cost usage failed provider=codex duration=1.3s error=Local token history is unavailable or incomplete.
Each pass reads the same number of bytes, so it is not resuming:
Codex cost scan applied work limits (bytesConsumed=412369328 deferredByBudget=0 deferredByTime=0 maxBytesPerRefresh=536870912 maxFileBytes=268435456 partialFiles=2)
Codex cost scan applied work limits (bytesConsumed=421551265 deferredByBudget=0 deferredByTime=0 maxBytesPerRefresh=536870912 maxFileBytes=268435456 partialFiles=2)
These two lines repeat with identical values.
This error also repeats several times per refresh, about 500 times in 6 hours:
cost-usage store operation failed; keeping database: sqlite(10)
There is 64 GB of free disk space.
Cache state (cost-usage.sqlite)
scan_metadata:catchUpPending = true,completedFiles = 86,totalFiles = 88- The stuck row in
files:size = 809825875,parsed_bytes = 268435456,anchor_indexed_bytes = 268435456,scan_complete = 0 - Its
scan_stateis about 2.5 MB. The decodedresumePayloadholds a truncated partial line:offset = 268435456lineStartOffset = 267753485lineBytes = 681971truncated = truejsonTailState.insideString = true,containerDepth = 6
- The token snapshots for this file cover only the first week of the session. Nothing after the cut-off is counted.
Likely trigger
The line containing the 256 MiB boundary is one "type":"compacted" record (with a replacement_history payload) that is about 5.0 MB long. It starts about 0.68 MB before the cap and ends about 4.3 MB after it. No line in the file is larger than 20 MB, and the lines around the boundary are ordinary.
My guess is one of these:
- Resuming in the middle of a truncated JSON line fails or is thrown away on each pass, so the file restarts from the same resume point every time.
- The large resume state (
scan_stateabout 2.5 MB,detailsPayloadabout 2 MB) fails to save. The repeatingsqlite(10)errors fit this. Then the next pass starts from the old state.
Expected behavior
- The scanner should get past the boundary line on a later pass and finish the file.
- Failing that, the app should show partial Codex data with a coverage warning. It should not fail the whole Codex cost report.
Related
- #2815: a large active rollout stays incomplete
- #3203: deferred large files starve
- #4140: a stalled catch-up blocks model breakdowns
These are closed. This looks like a remaining case where the 256 MiB cut-off splits a large line.
I can share more cache fields or a byte-range description of the boundary line on request. I can't share the session contents.
- Dominant language
- Swift
- Stars
- 23.3k
- Forks
- 2.2k
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 231
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from steipete/CodexBar
-
clawsweeper:needs-maintainer-review clawsweeper:no-new-fix-pr impact:other issue-rating: 🦪 silver shellfish P2
Difficulty 1/5 Under an hour Newbie friendliness 72/100
steipete/CodexBar#5147 · 1 comment · 1 reaction ·
Maintainers usually reply within 1 day
-
googledeveloper: unclaimed Ultra credit (null remaining amount) fails with "unrecognized response" instead of "Not applied"Possibly taken @djbclark claimed this today. Openclawsweeper:linked-pr-open clawsweeper:no-new-fix-pr clawsweeper:source-repro impact:ux-friction issue-rating: 🦞 diamond lobster P2
Difficulty 2/5 1-3 hours Newbie friendliness 28/100
steipete/CodexBar#5145 · 1 comment · 1 reaction ·
Maintainers usually reply within 1 day
-
clawsweeper:needs-maintainer-review clawsweeper:no-new-fix-pr impact:other issue-rating: 🦪 silver shellfish P2
Difficulty 3/5 1-2 days Newbie friendliness 50/100
steipete/CodexBar#5144 · 1 comment · 1 reaction ·
Maintainers usually reply within 1 day
-
Claude CLI source drops "Current week (all models)" when /usage redraws over earlier screen contentOpenclawsweeper:fix-shape-clear clawsweeper:queueable-fix clawsweeper:source-repro impact:ux-friction issue-rating: 🦞 diamond lobster no-stale P2
Difficulty 4/5 3-5 days Newbie friendliness 30/100
steipete/CodexBar#5143 · 1 comment · 1 reaction ·
Maintainers usually reply within 1 day
-
issue-rating: 🌊 off-meta tidepool
Difficulty 1/5 Under an hour Newbie friendliness 1/100
steipete/CodexBar#5131 · 1 comment · 1 reaction ·
Maintainers usually reply within 1 day
All issues in steipete/CodexBar
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
nwverse/persona#12800 · 1 comment ·
Maintainers usually reply within 1 day
-
bug documentation
Difficulty 1/5 Under an hour Newbie friendliness 86/100
ajithrnayak/xpunge#7 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
DuarteSantos8/openGym#1469 ·
Maintainers usually reply within 2 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 80/100
Maintainers usually reply within 1 day