[PowerSync] A synced row that fails deserializationSchema hangs pending writes and silently stops sync for the table
I maintainer di solito rispondono entro 1 giorno
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Idoneità per principianti
- 48/100
- Tipo di issue
- Bug
- Chiarezza
- Specificata chiaramente
- Stato di attività
- Attiva
- Stack tecnologico
- sqlite, typescript
Direzione di ricerca
Read packages/powersync-db-collection/src/powersync.ts, especially flushDiffRecordsWithContext and its catch near line 568; the issue describes the failure path and a Node reproduction. Trace how pending operations are resolved and how collection failures are reported, then run the provided reproduction with a malformed row. Done when the write settles or rejects, later rows are handled, and reload reports a visible error or becomes ready.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
- I've validated the bug against the latest version of DB packages
Describe the bug
In @tanstack/powersync-db-collection, one synced row that the deserializationSchema rejects stops the table and leaves local writes hanging.
With the collection already ready:
- The bad row reaches SQLite.
onDeserializationErrorfires and the adapter logsAn error has been detected in the sync handler. So far as documented. - The collection stays
readyandhealth.hasErrorsstaysfalse, but it applies nothing more for that table: later changes to other rows never show.onDeserializationErrorfires again on every later flush for the same row. - A local write through the collection reaches
ps_crud, buttx.when('settled')neither resolves nor rejects. - Creating the collection again over the same database (a reload) fails its first load, so it never becomes ready.
- Only fixing the row and reloading recovers it.
Cause, read from packages/powersync-db-collection/src/powersync.ts on main (same code in 1.0.0-rc.1.1 and 1.0.0-rc.1.3):
flushDiffRecordsWithContextcallsdeserializeSyncRow()for each tracked diff record, which callsonDeserializationErrorand throws.- The
catchat the end of that function (around line 568) only logs. So theDELETE FROM <tracking table>,commit()andpendingOperationStore.resolvePendingFor(pendingOperations)are never reached. - The next flush re-reads the same record first and throws again.
PowerSyncTransactor.applyTransactionwaits onwhenComplete, which onlyresolvePendingForresolves, so the local write never settles.- On startup, the same throw fails the first load.
The docs call a failed deserialization fatal, and onDeserializationError exists to react to it. The part that looks wrong is that the failure isn't surfaced through fail() / reportError() (#2168), and a pending write neither settles nor rejects.
To Reproduce
Versions: @tanstack/db 1.0.0-rc.1.3, @tanstack/powersync-db-collection 1.0.0-rc.1.3, @powersync/node 1.3.0 (@powersync/common 2.5.0), better-sqlite3, zod 4.6.7, Node 25. No sync service is needed: a local SQL write stands in for a synced row.
import { mkdtempSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { PowerSyncDatabase, Schema, Table, column } from '@powersync/node'
import { createCollection } from '@tanstack/db'
import { powerSyncCollectionOptions } from '@tanstack/powersync-db-collection'
import { z } from 'zod'
const sleep = (ms) => new Promise((r) => setTimeout(r, ms))
const race = (p, ms) =>
Promise.race([
p.then(() => 'settled', (e) => `rejected: ${e?.message ?? e}`),
sleep(ms).then(() => `still pending after ${ms / 1000} s`),
])
const AppSchema = new Schema({ todos: new Table({ title: column.text, meta: column.text }) })
const db = new PowerSyncDatabase({
schema: AppSchema,
database: { dbFilename: 'repro.sqlite', dbLocation: mkdtempSync(join(tmpdir(), 'ps-deser-')) },
})
await db.init()
const schema = z.object({ id: z.string(), title: z.string().nullable(), meta: z.record(z.string(), z.unknown()).nullable() })
const deserializationSchema = z.object({
id: z.string(),
title: z.string().nullable(),
meta: z.string().nullable().transform((v, ctx) => {
if (v == null) return null
try {
return JSON.parse(v)
} catch {
ctx.addIssue({ code: 'custom', message: `meta is not JSON: ${v}` })
return z.NEVER
}
}),
})
let deserErrors = 0
const make = (id) =>
createCollection(
powerSyncCollectionOptions({
id,
database: db,
table: AppSchema.props.todos,
schema,
deserializationSchema,
onDeserializationError: () => deserErrors++,
}),
)
const todos = make('todos-1')
await todos.preload()
// 1. A row with malformed JSON in `meta`, as a synced row would arrive.
await db.execute(`INSERT INTO todos (id, title, meta) VALUES ('bad', 'bad', '{"a":')`)
await sleep(500)
// 2. One local write.
const tx = todos.insert({ id: 'good', title: 'good', meta: { ok: true } })
console.log('2.', await race(tx.when('settled'), 15_000))
// 3. A later change to another row.
await db.execute(`INSERT INTO todos (id, title, meta) VALUES ('later', 'later', '{}')`)
await sleep(500)
console.log('3.', todos.has('later'), todos.status, todos.health.hasErrors, deserErrors)
// 4. A reload: a second collection over the same database.
const again = make('todos-2')
console.log('4.', await race(again.preload(), 10_000), again.status)
process.exit(0)
Output (adapter log lines trimmed):
2. still pending after 15 s
3. false ready false 3
4. rejected: Failed to validate incoming data for todos. Issues: meta - meta is not JSON: {"a": error
Control: the same script without the bad insert prints 2. settled, 3. true ready false 0 and 4. settled ready.
We first hit this in a browser against a real PowerSync service (@powersync/web 2.6.0; Chromium, WebKit, Firefox and Brave under Playwright). The pending write never settled and reloads never became ready in every run: 14 of 14 on rc.1.1, 4 of 4 on rc.1.3.
Expected behavior
- A pending write settles or rejects. It never hangs.
- A row that fails
deserializationSchemais either reported and skipped (its tracking record deleted, the rest of the batch committed), or the failure is surfaced through the Collection withreportError()/fail(), sohealthorstatusshows it. Todaystatusstaysreadyandhealthshows nothing while the table stops applying changes. - A reload with the bad row in place either becomes ready with the row reported, or fails with a visible error. Today the second preload rejects and the collection stays in
error.
Fix pointer: the catch at the end of flushDiffRecordsWithContext. Deserializing each record under its own try/catch, and skipping and reporting the bad one, lets the DELETE, commit() and resolvePendingFor run. The alternative is to reject the pending operations and call fail() / reportError() there.
Workaround we use: a deserializationSchema that never throws. The transform catches its own error and returns a marker value, and the app counts the marked row and leaves it out of the UI. With that, the same browser histories pass every run: the bad row shows as a problem, every write settles, and reloads become ready. It gives up onDeserializationError for those columns.
Desktop
- OS: macOS
- Browser: Chromium, WebKit, Firefox, Brave (Playwright), plus the Node repro above
- Version:
@tanstack/powersync-db-collection1.0.0-rc.1.3 (also 1.0.0-rc.1.1)
Additional context
Related:
- #2168 (
reportError/failafter ready). - #2118: no conformance law covers a delivery that the adapter's own schema rejects. The PowerSync driver declares no
delivery-errorcapability and nodeserializationSchema, so the oracle never generates this case. - #1647: a different cause with the same shape, a pending operation that
resolvePendingFornever sees.
- Lingua principale
- TypeScript
- Stelle
- 4k
- Fork
- 277
- Merge medio
- 1g 11h
- PR unite (30g)
- 148
Preparare l'ambiente
- Nessun Dockerfile né file Docker Compose
- Ha un modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di TanStack/db
-
Difficoltà 1/5 1-3 ore Idoneità per principianti 88/100
I maintainer di solito rispondono entro 1 giorno
-
OfflineExecutor.dispose() never settles pendingTransactionPromisesForse di nuovo libera @Claire-ovo l’ha presa 10 giorni fa e non c’è nessuna pull request aperta. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
TanStack/db#2254 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
I maintainer di solito rispondono entro 1 giorno
-
Two @tanstack/db tests time out under CPU load because a cold module import counts against the 5 s default timeoutForse già presa @jp-knj l’ha presa 8 giorni fa. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 48/100
I maintainer di solito rispondono entro 1 giorno
Issue simili
-
area: web bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100
kinqsradio/orbyn-engineering#227 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 85/100
joeldesante/TurfBuilder#258 ·
I maintainer di solito rispondono entro 1 giorno
-
bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
ncecere/open-mcp-portal#85 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 63/100
loathers/garbage-collector#2865 ·
I maintainer di solito rispondono entro 1 giorno
-
test-reliability
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
Azure/azure-sdk-for-js#40672 · 1 reazione ·
I maintainer di solito rispondono entro 1 giorno