feat: Do not count a removed session's cost again when an import brings it back
Maintainer thường phản hồi trong vòng 1 ngày
@huang195 đang làm issue này rồi.
Từ ngày 16/11/2026.
Đánh giá
Issue này chưa được đánh giá.
Mô tả
Part of #1871 (PR3). Builds on #1876 (merged). It does not depend on PR2, so it can run next to PR2.
Problem
The cost ledger has no session id. Its rows are totals per minute and per model, agent and project. So Cortex cannot tell from the ledger which turns of a session it already counted.
An import must bring back a session that the archive no longer holds, and the session must come back in full. But when the archive forgot that session, its cost rows stayed in the ledger. Today the import then sends the session's turns to the ledger again. The same cost is counted twice in the today, 7d and month figures.
The archive forgets a session in four ways. All four leave the ledger rows in place:
| Way | Code | When |
|---|---|---|
x in agentop (DELETE /v1/sessions/{id}) |
archive/delete.go deleteSession |
the user removes one session |
Clear (DELETE /v1/sessions) |
archive/clear.go clearAll |
the user removes all sessions |
| Day limit | archive/archive.go prune |
30 days after the session's last write |
| Size limit | archive/archive.go prune |
the archive is over max_bytes |
Rule: a removed session comes back on import, and its cost is not counted again. The ledger does not change on a remove (it holds what was really spent).
Design
What Cortex keeps
When the archive forgets a session, it writes one forget record: the time span of that session's turns that the ledger may already hold.
{"id":"a0ff48c4-5287-4062-ab7a-690799c406ad","from":1794700000000,"to":1794793835708}
id: the session id.from: the session's first event time (sessionMeta.CreatedAt), as Unix milliseconds. For an imported session,WriteSessionsets this to the first transcript event (archive/import.go:170). For a live session, it is the first live event.to: the session's last request time (SessionSummary.LastRequestAt), as Unix milliseconds.
Why a span and not only to: a live session can have transcript turns from before Cortex was installed. Those turns are older than from and were never in the ledger, so they must still be sent.
The builder checks first that pruning a session's oldest segments does not change CreatedAt. If it does, the record must take from from the first time the session was archived.
Write a record only for a session with ledger rows: LastRequestAt is set and the summary has tokens or cost. On this laptop, 457 of 2,001 sessions (23%) have no tokens and no cost (git, curl and similar traffic). They need no record.
Where it is kept
<archive root>/forgotten.jsonl, next todata/, not inside it.clearAllrenames onlydata/, so the file survives a clear. A clear must write a record for each session it removes.- One JSON line per record, appended. The archive's writer goroutine is the only writer, so no lock is needed.
- Write the record before the session is removed, and fsync it. A crash between the two then leaves an extra record for a session that is still in the archive. That is harmless, because the import looks at the archive first. The other order could lose the record and cause a double count.
- At
Open, the archive reads the file into a mapid -> {from, to}. A second record for the same id merges: the smallestfromand the largestto. - A line that does not parse is skipped and counted (
Stats). It does not stopOpen.
How long a record is kept
A record is needed only while the ledger can still hold rows for its turns. The ledger keeps rows for cost_ledger.retention_days (default 31), counted by turn date, and the importer already skips every turn older than the ledger's cutoff (RetentionCutoff). So:
- Drop a record when
tois older than the ledger cutoff minus 1 day. The extra day covers the ledger's whole-day files. - The archive gets the ledger retention at
Open(a new option, for exampleWithLedgerRetentionDays(n)).cmd/cortexpassescost_ledger.retention_days, or the ledger default. With the ledger off, it passes 0 and the archive writes no records. - On
Open, and once a day on the archive's existingtick, rewrite the file without the expired records (temp file, fsync, rename), when at least half of its lines are dead. - A change of
retention_daysstays safe:- Higher: the ledger has already pruned the older rows, so a turn whose record expired is not in the ledger either.
- Lower: records are kept longer than needed. That is harmless.
How much this is, over months or years
The file does not grow with the time Cortex runs. It holds at most the sessions forgotten within the ledger retention plus 1 day.
Measured on one heavy laptop (2026-10-17 to 2026-11-16): 2,001 sessions in 31 days, so 65 a day on average and 638 on the busiest day. Ids are 38 characters on average (45 maximum). One record is about 80 bytes on disk and about 200 bytes in memory.
| Case | Records alive | Disk | Memory |
|---|---|---|---|
| Only the day limit forgets (normal use). The archive forgets a session about 30 days after its last write, and the record expires about 2 days later | about 2 days of sessions: about 130 | about 10 KB | about 26 KB |
| The user clears everything every day, with default retention | 32 × 65 = about 2,100 | about 170 KB | about 420 KB |
| Same, at the busiest rate seen (638 a day) | about 20,400 | about 1.6 MB | about 4 MB |
retention_days: 3650 (the maximum), clear every day, 65 a day |
about 237,000 | about 19 MB | about 47 MB |
Only the last row is large. It needs both the 10-year ledger maximum and a full clear every day. For it, the builder may hold the map as id -> [2]int64 with ids interned, but that is not required.
How the import uses it
In importer.Run, for a session that is not in the archive:
- Is there a forget record? Then send to the ledger only the turns with a request time before
fromor afterto, and still none older than the ledger cutoff. - No record: send every turn within the ledger cutoff (today's rule), with today's guess by time and project for live sessions.
The guess stays for sessions forgotten before this change, which have no record. 32 days after an install upgrades, every such session is past the ledger cutoff and the guess has nothing left to do. Leave a shortcut: comment saying it can go in a later release.
The rules for a session that is in the archive do not change:
- Live: skip.
- Imported: replace, and send only turns after its
LastRequestAt.
WriteSession's own delete for a replace must not write a record.
New archive method, about: Forgotten(id string) (from, to time.Time, ok bool). It reads the in-memory map.
What does not change
- The cost ledger: no new field, no rewrite, no negative rows. A remove keeps the real spend.
- The session API, agentop and the import report.
Tests
- Each of the four forget paths writes a record with the right
fromandto, and a session with no inference writes none. - A clear writes one record per session, and
forgotten.jsonlsurvives the clear. - Remove an imported session with
xand import again: the session is back in full, and the ledger request count and cost are unchanged. - The same for a live session whose transcript has 2 turns from before its first live event: those 2 turns reach the ledger, and the live turns do not reach it again.
- Remove, then add turns to the transcript, then import: only the new turns reach the ledger.
- An archive that prunes a session by the day limit: re-import sends no turn twice.
- Records expire after the ledger cutoff plus 1 day, and the compaction keeps the live ones.
- A record written before a crash, with the session still on disk: the import follows the archive, not the record.
- A corrupt line in
forgotten.jsonlis skipped and counted. - With the ledger off, no records are written.
Acceptance criteria
- A session that
x, a clear, the day limit or the size limit removed comes back in full on import. - Its cost is not counted again in today, 7d or month. Turns added after the remove are counted once.
-
forgotten.jsonlstays bounded by the ledger retention, not by how long Cortex runs. -
go vetand-racetests pass forcore/session/archive,core/session/importerandcmd/cortex.
For a human after review
- On a laptop install: import, remove a few sessions with
x, import again, and check thatagentop cost --window monthdoes not change.
- Ngôn ngữ chính
- Go
- Star
- 26
- Fork
- 46
- Merge trung bình
- 9 giờ 55 phút
- Pull request đã merge (30 ngày)
- 257
Chuẩn bị môi trường
- Không có Dockerfile hay tệp Docker Compose
- Không có mẫu pull request
- Đọc hướng dẫn đóng góp
Bắt đầu từ đâu
- Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
- Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
- Fork repository và làm thay đổi trên một nhánh.
- Mở pull request có tham chiếu số hiệu của issue.
Issue khác của rossoctl/cortex
-
bug laptop
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 75/100
Maintainer thường phản hồi trong vòng 1 ngày
-
bug laptop
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 78/100
Maintainer thường phản hồi trong vòng 1 ngày
-
bug: The JWT bypass matches a cleaned path but forwards the raw path, so /mcp/../healthz skips validationCó thể đã có người làm @evaline-ju đã nhận hôm nay. Đang mởbug
rossoctl/cortex#1884 · 1 người được giao ·
Maintainer thường phản hồi trong vòng 1 ngày
-
bug: One-hour Anthropic cache writes are priced at the 5-minute rate, so spend is 37.5% too lowĐang mởbug
Độ khó 3/5 1-2 ngày Mức phù hợp với người mới 78/100
Maintainer thường phản hồi trong vòng 1 ngày
-
bug laptop
Độ khó 3/5 1-2 ngày Mức phù hợp với người mới 68/100
Maintainer thường phản hồi trong vòng 1 ngày
Tất cả issue của rossoctl/cortex
Issue tương tự
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 65/100
Containerpak/cpak#93 ·
-
bug triage
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 72/100
Maintainer thường phản hồi trong vòng 1 ngày
-
Độ khó 1/5 Dưới một giờ Mức phù hợp với người mới 82/100
router-for-me/CLIProxyAPI#7607 ·
Maintainer thường phản hồi trong vòng 1 ngày
-
enhancement ui/ux
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 82/100
myNameArnav/lex#52 ·
Maintainer thường phản hồi trong vòng 1 ngày
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 64/100