Priority: P2 — flash durability & corruption-recovery audit for wallet.db and new durable stores (G06/G09 add writes; this issue bounds the write cost and the blast radius).
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 28/100
Research direction
Start by reading gonuts storage/bolt.go and its bbolt version and open options; then inspect src/cmd/tollgate-cli/ and the named persistent stores. The work also depends on G06/G09 landing before their stores can be included. Done means the audit table, measured write-amplification figures, recovery runbook, wallet check tests, and power-loss campaign report meet the listed acceptance criteria.
Written by the indexing model from the issue text.
Description
Priority: P2 — flash durability & corruption-recovery audit for wallet.db and new durable stores (G06/G09 add writes; this issue bounds the write cost and the blast radius).
Problem
The wallet (bbolt), the drain journal, and (if G06/G09 land) session/payment stores all write to router flash. Unknown/uncontrolled today: fsync semantics of bbolt on OpenWrt kernels (default NoSync=false? options at open), write amplification of bbolt B+tree rewrites vs the flash budget (8 MB flash routers have limited erase cycles), power-loss behavior of each file (bbolt is designed for crash safety with fsync — verify our open flags don't disable it), and corruption detection/repair (bbolt ships bolt check — not exposed anywhere).
Why it matters
Fund-relevant state on cheap NAND with power cuts is the environment; "it worked in Docker" is not evidence. A torn bbolt after power loss = potentially the whole wallet (counter lineage + proofs). Also SD-card-class storage (some deployments) silently corrupts; detection tooling is the difference between scheduled recovery and surprise loss.
Current behavior (source refs)
- gonuts
wallet/storage/bolt.go—bolt.Open(filepath.Join(path, "wallet.db"), 0600, nil)— default options (fsync on commit — verify against bbolt version vendored); noNoGrowSync/NoSync— good if defaults hold; prove it. - Drain journal (#433): fsync'd append — good precedent.
- No
bolt check/repair tooling exposed; no backup guidance beyond MIGRATION docs.
Desired invariant
Every fund-relevant file either survives power loss with its guarantees intact (fsync'd writes, atomic renames) or is reconstructable (journaled + documented repair path); steady-state write amplification fits the flash budget of supported devices (measured, not assumed).
Proposed scope
- Audit write paths + flags for every persistent store (bbolt open flags per cgo-free bbolt fork vendored in gonuts; sessions JSON tmp+rename; drain journal; G09 store).
- Measure: bytes-written per payment/payout/session-update on lab hardware (
/proc/<pid>/iodeltas over N ops) for both metrics; publish numbers; set budgets. - Corruption tooling:
tollgate-cli wallet checkwrapping bbolt consistency check; documented recovery runbook (backup, check, export tokens, re-init) — links to the migration tooling. - Power-loss campaign (PRTA smart-plug or QEMU
quit): N=100 cut-during-write cycles on wallet.db → zero silent corruption (post-bootcheck+ balance assertion).
Areas / files
gonuts storage/bolt.go, TollGate src/cmd/tollgate-cli/, src/valve/src/merchant stores (if G06/G09 landed), PRTA lanes, docs (runbook).
Acceptance criteria
- Audit table (file, write path, fsync, atomicity, repair) merged.
- Write-amplification numbers published; power-loss campaign report with zero silent corruptions (or found issues fixed and re-run).
Required tests
- The campaign IS the test; plus
wallet checkunit on seeded corrupt DBs.
Failure-injection tests
- Power-cut distribution across commit/compact/rename points (automated).
Compatibility
None (additive tooling + docs).
Dependencies
G06/G09 land first for their stores to be included; wallet.db audit independent.
Out of scope
- Switching storage engine.
- Dominant language
- Go
- Stars
- 12
- Forks
- 14
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 217
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from OpenTollGate/tollgate-module-basic-go
-
go-battery needs an ndsctl on PATH: TestPurchaseSessionGuardHoldsThroughTheOutcomeUnknownWindow fails on bare hosts (passes with stub)Possibly taken @Amperstrand claimed this 1 day ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
OpenTollGate/tollgate-module-basic-go#726 · 2 comments ·
Maintainers usually reply within 1 day
-
rebrand-literal-gutter: uhttpd section-vocabulary check trips on a COMMENT (uhttpd.luci in 92-tollgate-admin-setup:178)Possibly taken @Amperstrand claimed this 1 day ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
OpenTollGate/tollgate-module-basic-go#723 ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
OpenTollGate/tollgate-module-basic-go#768 ·
Maintainers usually reply within 1 day
-
Four drift fences for tests/contract/ (+ test.yml clean-container lane + pre-commit wiring)Possibly taken A pull request linked to this issue is open or already merged. Open
Difficulty 5/5 Over a week Newbie friendliness 25/100
OpenTollGate/tollgate-module-basic-go#767 ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
OpenTollGate/tollgate-module-basic-go#763 ·
Maintainers usually reply within 1 day
All issues in OpenTollGate/tollgate-module-basic-go
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
-
[Chore] Remove dead AutogenV2 feature flagPossibly taken @geeknishantkyeus claimed this today. Openbug triage
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
kyverno/kyverno#17936 · 1 comment · 1 reaction ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100