compact: build the successor in a private directory and publish it atomically

Aperta
  • Qualcosa non va?
#59 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
35/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
javascript, node.js
Ambito
backend, databases

Direzione di ricerca

Start with okf/faces/lib/compact.js and read guardCompactPaths(), claimDir(), and the staging and cleanup flow. The issue also identifies Bundle.init and the sign callback as key points to trace. Add the three listed race tests, then verify that compaction refuses a changed destination and cleanup preserves replacement directories and files.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Summary

compact() in okf/faces/lib/compact.js (branch feat/okf-autobee-v2) checks its paths once, then writes into them in place. Another process that writes into the out-dir or the staging path while a compaction runs can lose its files. The review of the compaction hardening (T13 to T18) found two such races. The owner accepted both for the first release, as a fast follow after the branch lands.

The two races

  1. The out-dir check expires before the store opens. guardCompactPaths() refuses a non-empty out-dir before the face step. Compaction then materializes, stages and awaits the sign callback. If another process creates the out-dir and writes a file into it during that window, claimDir() accepts the directory, and Bundle.init (Corestore) clears it. The foreign file is gone, and compaction reports success.
  2. Cleanup equates a path name with ownership. On a failure, compaction removes the stage when ownStage is true, and claimDir().release() removes the out-dir or the entries that it did not see at claim time. If another process replaced the stage directory, or added a file to the out-dir after the claim, cleanup deletes that directory or file.

Both races need a concurrent writer on the same paths. A single user on one machine who runs one compaction at a time does not meet them.

The full fix

  • Build the successor in a private directory that compaction creates exclusively, for example mkdtemp(<out-dir>.compact-build-), beside the out-dir.
  • Stage, ingest and verify there. Close the successor vault.
  • Publish with an atomic rename onto the out-dir. On POSIX, rename(2) of a directory fails with ENOTEMPTY or EEXIST when the destination is not empty, so a changed destination is refused, never cleared.
  • Reopen the published vault at the out-dir and return it. compact() then returns a vault that it reopened, not the one that it built. That is an API change for callers that hold the build path.
  • Cleanup removes only the private build directory and the stage that this run created. Record the device and inode of each created directory, and refuse to remove a path whose identity changed.

Tests to add

  • A sign callback that creates the out-dir and writes a file into it: compaction refuses, and the file stays.
  • A stage replaced between staging and a failure: the replacement stays.
  • A file added to the out-dir after the claim, then an init failure: the file stays.
Lingua principale
JavaScript
Stelle
0
Fork
0
Nessuna PR unita negli ultimi 30g

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di indexzero/hyperdown

Tutte le issue di indexzero/hyperdown

Issue simili

Altre issue su JavaScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.