[BUG]: syncToFs() resolves immediately, without syncing, when another sync is already queued
I maintainer di solito rispondono entro 1 giorno
Valutazione
- Difficoltà
- 3/5
- Tempo stimato
- 1-2 giorni
- Idoneità per principianti
- 35/100
- Tipo di issue
- Bug
- Chiarezza
- Specificata chiaramente
- Stato di attività
- Ferma
- Stack tecnologico
- postgresql, typescript, wasm
- Ambito
- backend-api-design, databases
Direzione di ricerca
Start by tracing syncToFs() and its mutex handling; the issue provides a reproduction using a slow MemoryFS and the expected ordering. Run that reproduction and inspect the existing sync tests, if present, to find where to cover calls made while a sync is queued. Done means each caller resolves only after a sync that started after its call completes. An open linked pull request (#1193) indicates work is already underway.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Describe the bug
syncToFs() coalesces concurrent calls with a pending flag:
async syncToFs() {
if (pending) return;
pending = true;
…
await mutex.runExclusive(() => {
pending = false;
return fs.syncToFs(…)
});
}
pending is set when a sync is requested and cleared only when it starts. A call made while a sync is requested but not yet started therefore returns at once, without awaiting anything. Its writes will be persisted by that queued sync, but its promise has already resolved before the queued sync has even begun.
With relaxedDurability: false, an awaited syncToFs() - and every top-level query()/exec(), which end with it - is expected to be a durability boundary. Under concurrent sync requests, it isn't: the caller resolves before any sync that started after its call. This also affects #1066, which makes transaction() end with this.syncToFs(): a commit that races with a queued sync would still resolve before it is persisted.
To Reproduce — include code sample(s)
A MemoryFS with a slow syncToFs() stands in for a persistence layer such as idb://:
import { PGlite, MemoryFS } from "@electric-sql/pglite";
class SlowFS extends MemoryFS {
log = [];
async syncToFs(relaxed) {
const n = this.log.filter((e) => e.startsWith("start")).length + 1;
this.log.push(`start ${n}`);
await new Promise((r) => setTimeout(r, 50));
await super.syncToFs(relaxed);
this.log.push(`end ${n}`);
}
}
const fs = new SlowFS();
const pg = await PGlite.create({ fs });
await pg.exec("CREATE TABLE t (v int)");
fs.log.length = 0;
const a = pg.syncToFs().then(() => fs.log.push("A resolved"));
await new Promise((r) => setTimeout(r, 10)); // sync 1 is running
const b = pg.syncToFs().then(() => fs.log.push("B resolved")); // queued behind sync 1
const c = pg.syncToFs().then(() => fs.log.push("C resolved")); // requested while B's sync is queued
await Promise.all([a, b, c]);
await new Promise((r) => setTimeout(r, 200));
console.log(JSON.stringify(fs.log));
await pg.close();
Output:
["start 1","C resolved","end 1","start 2","A resolved","end 2","B resolved"]
C resolves while sync 1 is still running and before sync 2 - the first sync that can carry its writes - has started.
Expected: C resolves only after a sync that started after its call (here: after end 2), e.g. by returning the queued sync's promise instead of undefined.
Logs
None beyond the output above.
Details
- PGlite version: 0.5.4 and 0.5.15 (same output; the code path is unchanged)
- using any extensions? no (the repro uses none; our app uses
uuid_ossp) - OS version: Linux (CachyOS, kernel 6.18)
- node, bun, deno or browser version: Bun 1.4.3 for the repro; observed in the app on Chromium
Additional context
Found while working around #1065. In our app, the data loss we saw was fully explained by #1065; we have not reproduced a loss caused by this race on its own. The repro above shows the broken guarantee at the API level. On our side we serialize syncToFs() so that each caller gets the promise of a sync that starts after its call, and callers that arrive while one is still queued share it.
- Lingua principale
- TypeScript
- Stelle
- 16.3k
- Fork
- 474
- Merge medio
- 23h 53m
- PR unite (30g)
- 53
Preparare l'ambiente
Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di electric-sql/pglite
-
Feature request: Nitro 3 support (`unwasm` export condition)Forse di nuovo libera Una pull request per questa issue è stata chiusa senza essere unita. Aperta
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 85/100
electric-sql/pglite#1120 · 16 commenti ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 5/5 Più di una settimana Idoneità per principianti 25/100
electric-sql/pglite#1200 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 54/100
electric-sql/pglite#1192 ·
I maintainer di solito rispondono entro 1 giorno
-
Feature request: live: allow pausing the notify triggers during bulk writes (e.g. pglite-sync applies) and refresh once on resumeForse già presa @tdrz l’ha presa 13 giorni fa. Apertaenhancement
Difficoltà 4/5 3-5 giorni Idoneità per principianti 25/100
electric-sql/pglite#1184 · 2 commenti · 1 assegnatario ·
I maintainer di solito rispondono entro 1 giorno
-
pglite-socket: handleData copies the whole buffer for every chunk (quadratic time for large messages)Forse già presa @benclmnt l’ha presa 10 giorni fa. Aperta
Difficoltà 3/5 1-2 giorni Idoneità per principianti 62/100
electric-sql/pglite#1181 · 3 commenti ·
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di electric-sql/pglite
Issue simili
-
Bug: Broken RTL / BiDi text rendering when mixing Persian (Farsi) with English or numbersForse già presa Una pull request collegata a questa issue è aperta o già unita. Apertabug
Difficoltà 2/5 Mezza giornata Idoneità per principianti 68/100
amosblomqvist/learnharness#758 ·
I maintainer di solito rispondono entro 2 giorni
-
Change WorkspacesAperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 62/100
bitfocus/companion-module-renewedvision-propresenter-api#134 ·
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
rcpch/digital-growth-charts-react-component-library#304 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
dsh-market/dsh-market#1210 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
good first issue
Difficoltà 2/5 1-3 ore Idoneità per principianti 86/100
trustjonathan/STUDY-HUB#13 ·