WriteBufferManager stall: runtime allowStall mutation + late-CF size-0 bypass the open-time history clamp
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 45/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- cpp, nodejs, typescript
Research direction
Read AGENTS.md invariant 10, then inspect src/binding/database/db_settings.cpp around buildColumnFamilyOptions and resolveMaxWriteBufferSizeToMaintain. Run test/write-buffer-manager-stall.test.ts and trace the db.put() helper around its store.putSync() call. Done means the two reported mutation and late-CF cases are covered without permanent stalls, and the regression test fails within the test runner's deadline.
Written by the indexing model from the issue text.
Description
Summary
A Codex review on #758 (raised by @kriszyp) surfaced two concerns in the write-buffer-manager / retained-history feature (AGENTS.md invariant 10 — "Retained memtable history must fit the WriteBufferManager budget"). They are unrelated to that PR's background-error change (it only inherits these files via a main merge), so filing them here to track separately.
1. Runtime writeBufferManagerAllowStall mutation bypasses the open-time history clamp
src/binding/database/db_settings.cpp (~L209)
buildColumnFamilyOptions resolves the derived (-1) max_write_buffer_size_to_maintain to 0 only when a stalling manager is configured at open time (resolveMaxWriteBufferSizeToMaintain). A database can attach a WriteBufferManager while allowStall is false, so its column families keep RocksDB's derived (large) history target. Flipping writeBufferManagerAllowStall to true at runtime then enables stalling on that same manager without revisiting those already-resolved CF targets. With a budget below the retained history, every write to that database can stall permanently — the exact failure invariant 10 exists to prevent.
Related size-0 mismatch for late-created families: an existing DB retains the manager, while resolveMaxWriteBufferSizeToMaintain() consults the global size and may re-derive a large history for a CF created after the global size was reset to 0.
Suggested direction: enforce the invariant against the manager actually attached to each descriptor — either make allowStall immutable after attachment, or safely update/reject incompatible CF max_write_buffer_size_to_maintain targets before enabling stall.
Coverage gap: add tests for (a) enabling stall after open, and (b) creating a CF after the global size is reset to 0.
2. Stall regression test can hang instead of failing
test/write-buffer-manager-stall.test.ts (~L41)
db.put() is declared async but calls store.putSync() before returning its promise. If RocksDB stalls inside that native call, the JS worker thread is blocked and Vitest's 60s timeout can't fire — the regression hangs until the outer CI job timeout rather than failing this test.
Suggested direction: run the stall scenario in a worker/child process the parent can terminate on a deadline, or drive it through a genuinely asynchronous native operation so the runner can still enforce its timeout.
Reported by @kriszyp's Codex review on #758; filed as a scoped follow-up against the write-buffer-manager work.
- Dominant language
- C++
- Stars
- 24
- Forks
- 3
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 49
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from HarperFast/rocksdb-js
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
HarperFast/rocksdb-js#945 ·
Maintainers usually reply within 1 day
-
Drop Deno from CI and README (keep Deno-specific fixes)Possibly taken @cb1kenobi claimed this 1 day ago. Open
HarperFast/rocksdb-js#947 · 1 assignee ·
Maintainers usually reply within 1 day
-
Difficulty 4/5 1-2 days Newbie friendliness 45/100
HarperFast/rocksdb-js#946 ·
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
HarperFast/rocksdb-js#944 ·
Maintainers usually reply within 1 day
-
Durable-handle restore replays onto a partly rebuilt transaction; a failed capture leaves it unresetOpen
Difficulty 4/5 3-5 days Newbie friendliness 48/100
HarperFast/rocksdb-js#937 ·
Maintainers usually reply within 1 day
All issues in HarperFast/rocksdb-js
Similar issues
-
v9.6.0: CMake incorrectly enforces presence of libevdevPossibly taken A pull request linked to this issue is open or already merged. Open
Difficulty 2/5 1-3 hours Newbie friendliness 67/100
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 65/100
mltframework/shotcut#1943 ·
Maintainers usually reply within 1 day
-
Rowan (2017) electron heating gives NaN for beta > beta_max; clip() then sets electrons to kel_minOpen
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
parthenon-hpc-lab/kharma#213 · 1 comment ·
Maintainers usually reply within 3 days
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
metasequoiaime/msime#9113 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
tone-3000/tone3000-plugin#257 · 1 comment ·
Maintainers usually reply within 1 day