daemon: the evidence store is one JSON string, so a store over ~512 MB fails every check with 'Invalid string length' (DB subset-dedupe, 6 checks on #133)
Maintainer antworten meist innerhalb von 1 Tag
Dieses Issue hat noch niemand übernommen.
Bewertung
- Schwierigkeit
- 4/5
- Geschätzter Aufwand
- 3-5 Tage
- Anfängerfreundlichkeit
- 45/100
- Issue-Typ
- Bug
- Klarheit
- Größtenteils klar
- Aktivitätsstatus
- Aktiv
- Tech-Stack
- javascript, node.js
Rechercherichtung
Start with oracle-gaps/evidence.mjs, where diskStore serializes and reads the entire evidence store, then trace its callers in oracle-gaps/check.mjs and oracle-gaps/daemon.mjs. Choose a storage approach that avoids V8's string length limit. Add the test-only size ceiling and e2e history described in the issue; done means the next daemon check completes when that ceiling is reached.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Beschreibung
Summary
The daemon's evidence store (diskStore, evidence.mjs) writes the whole store as one JSON.stringify string and reads it back as one string. V8 caps a string at about 512 MB, so a store past that size makes every later check end in an internal error:
RangeError: Invalid string length
at JSON.stringify (<anonymous>)
at Object.flush (oracle-gaps/evidence.mjs:98:37)
at checkIn (oracle-gaps/check.mjs:2836:24)
at async Object.check (oracle-gaps/check.mjs:1693:14)
at async oracle-gaps/daemon.mjs:525:20
The CLI reports oracle-gaps: error Invalid string length: a bug in oracle-gaps, not in your oracle and exits 3.
Trace
TanStack DB 21c52a07, packages/db --source src/query/subset-dedupe.ts (559 units), Arca pool tag r133-dbb4-all. Each variant ran a daemon sequence from an empty state: a first check, 3 unchanged rechecks and 3 one-line-edit rechecks.
| CLI | Result | Store size |
|---|---|---|
| main 97a4e7a8 | all 7 checks complete | 425 MB at the end |
| #133 head 9bb7c74a, Vitest cache off | the 3rd edit recheck (the 7th check) fails | 457 MB at the last good flush |
| #133 head, Vitest cache on (every check recomputes all units) | the 2nd unchanged recheck (the 3rd check) and every later check fail | 358 MB at the last good flush |
Composition
A store after 2 checks with the cache on (#133 head): 1,114 rows, 1 variant each. Each variant has about 481 inputs (52 KB of JSON) and a unit payload of about 83 KB.
Of the inputs, 261,508 are fs: keys (41 MB), 118,996 are env: keys (10 MB) and 22,662 are stat: keys. #133's every-read inputs add about 60% to a variant. Main already sits at 425 MB after 7 checks, so main reaches the ceiling a few edits later on the same suite.
Proposed fix (choose one)
- Write and read the store one row per line (JSONL), streamed, so no single string holds the whole store.
- Or store each distinct input list once, by its hash, and have variants name it, since most units share their
env:andnode_modulesinputs. - Or both.
A bound on the store's size is a separate design question. The retention law (bounded retention) covers memory, not this file.
Under P9 the fix needs an oracle-first witness: a test-only knob that lowers the store's size ceiling, so that a constructed e2e history reaches the state with a small store and requires that the next daemon check complete.
Authority
R13 (the daemon is the default) and P10 (an internal error is a CLI defect). Found while measuring #133 for M "Vitest cache off, measured".
This issue was written by Isaac.
- Vorherrschende Sprache
- JavaScript
- Sterne
- 5
- Forks
- 1
- Ø Merge
- 11 Std. 18 Min.
- Gemergte PRs (30 T.)
- 66
Entwicklungsumgebung
Dieses Projekt bietet weder Dev-Container noch Dockerfile noch Beitragsleitfaden – die Einrichtung liegt bei Ihnen. Beginnen Sie mit der README; die allgemeinen Schritte stehen in unserem Leitfaden für den ersten Beitrag.
Erste Schritte
- Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
- Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
- Forken Sie das Repository und arbeiten Sie in einem Branch.
- Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.
Mehr aus TanStack/oracles
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 78/100
Maintainer antworten meist innerhalb von 1 Tag
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 62/100
Maintainer antworten meist innerhalb von 1 Tag
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 72/100
Maintainer antworten meist innerhalb von 1 Tag
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 64/100
Maintainer antworten meist innerhalb von 1 Tag
-
Schwierigkeit 5/5 Über eine Woche Anfängerfreundlichkeit 25/100
Maintainer antworten meist innerhalb von 1 Tag
Alle Issues in TanStack/oracles
Ähnliche Issues
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 66/100
Maintainer antworten meist innerhalb von 1 Tag
-
bug
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 83/100
swc-project/swc-next-issues#532 ·
-
domain/checkout Platform(Default) priority/low testing/acceptance
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 76/100
Maintainer antworten meist innerhalb von 1 Tag
-
bug confirmed css v6
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 68/100
Maintainer antworten meist innerhalb von 1 Tag
-
check:passed logos:edit
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 66/100
Maintainer antworten meist innerhalb von 1 Tag