[PowerSync] A synced row that fails deserializationSchema hangs pending writes and silently stops sync for the table

Aperta
#2,358 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
48/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
sqlite, typescript
Ambito
backend, databases

Direzione di ricerca

Read packages/powersync-db-collection/src/powersync.ts, especially flushDiffRecordsWithContext and its catch near line 568; the issue describes the failure path and a Node reproduction. Trace how pending operations are resolved and how collection failures are reported, then run the provided reproduction with a malformed row. Done when the write settles or rejects, later rows are handled, and reload reports a visible error or becomes ready.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

  • I've validated the bug against the latest version of DB packages

Describe the bug

In @tanstack/powersync-db-collection, one synced row that the deserializationSchema rejects stops the table and leaves local writes hanging.

With the collection already ready:

  1. The bad row reaches SQLite. onDeserializationError fires and the adapter logs An error has been detected in the sync handler. So far as documented.
  2. The collection stays ready and health.hasErrors stays false, but it applies nothing more for that table: later changes to other rows never show. onDeserializationError fires again on every later flush for the same row.
  3. A local write through the collection reaches ps_crud, but tx.when('settled') neither resolves nor rejects.
  4. Creating the collection again over the same database (a reload) fails its first load, so it never becomes ready.
  5. Only fixing the row and reloading recovers it.

Cause, read from packages/powersync-db-collection/src/powersync.ts on main (same code in 1.0.0-rc.1.1 and 1.0.0-rc.1.3):

  • flushDiffRecordsWithContext calls deserializeSyncRow() for each tracked diff record, which calls onDeserializationError and throws.
  • The catch at the end of that function (around line 568) only logs. So the DELETE FROM <tracking table>, commit() and pendingOperationStore.resolvePendingFor(pendingOperations) are never reached.
  • The next flush re-reads the same record first and throws again.
  • PowerSyncTransactor.applyTransaction waits on whenComplete, which only resolvePendingFor resolves, so the local write never settles.
  • On startup, the same throw fails the first load.

The docs call a failed deserialization fatal, and onDeserializationError exists to react to it. The part that looks wrong is that the failure isn't surfaced through fail() / reportError() (#2168), and a pending write neither settles nor rejects.

To Reproduce

Versions: @tanstack/db 1.0.0-rc.1.3, @tanstack/powersync-db-collection 1.0.0-rc.1.3, @powersync/node 1.3.0 (@powersync/common 2.5.0), better-sqlite3, zod 4.6.7, Node 25. No sync service is needed: a local SQL write stands in for a synced row.

import { mkdtempSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { PowerSyncDatabase, Schema, Table, column } from '@powersync/node'
import { createCollection } from '@tanstack/db'
import { powerSyncCollectionOptions } from '@tanstack/powersync-db-collection'
import { z } from 'zod'

const sleep = (ms) => new Promise((r) => setTimeout(r, ms))
const race = (p, ms) =>
  Promise.race([
    p.then(() => 'settled', (e) => `rejected: ${e?.message ?? e}`),
    sleep(ms).then(() => `still pending after ${ms / 1000} s`),
  ])

const AppSchema = new Schema({ todos: new Table({ title: column.text, meta: column.text }) })
const db = new PowerSyncDatabase({
  schema: AppSchema,
  database: { dbFilename: 'repro.sqlite', dbLocation: mkdtempSync(join(tmpdir(), 'ps-deser-')) },
})
await db.init()

const schema = z.object({ id: z.string(), title: z.string().nullable(), meta: z.record(z.string(), z.unknown()).nullable() })
const deserializationSchema = z.object({
  id: z.string(),
  title: z.string().nullable(),
  meta: z.string().nullable().transform((v, ctx) => {
    if (v == null) return null
    try {
      return JSON.parse(v)
    } catch {
      ctx.addIssue({ code: 'custom', message: `meta is not JSON: ${v}` })
      return z.NEVER
    }
  }),
})

let deserErrors = 0
const make = (id) =>
  createCollection(
    powerSyncCollectionOptions({
      id,
      database: db,
      table: AppSchema.props.todos,
      schema,
      deserializationSchema,
      onDeserializationError: () => deserErrors++,
    }),
  )

const todos = make('todos-1')
await todos.preload()

// 1. A row with malformed JSON in `meta`, as a synced row would arrive.
await db.execute(`INSERT INTO todos (id, title, meta) VALUES ('bad', 'bad', '{"a":')`)
await sleep(500)

// 2. One local write.
const tx = todos.insert({ id: 'good', title: 'good', meta: { ok: true } })
console.log('2.', await race(tx.when('settled'), 15_000))

// 3. A later change to another row.
await db.execute(`INSERT INTO todos (id, title, meta) VALUES ('later', 'later', '{}')`)
await sleep(500)
console.log('3.', todos.has('later'), todos.status, todos.health.hasErrors, deserErrors)

// 4. A reload: a second collection over the same database.
const again = make('todos-2')
console.log('4.', await race(again.preload(), 10_000), again.status)
process.exit(0)

Output (adapter log lines trimmed):

2. still pending after 15 s
3. false ready false 3
4. rejected: Failed to validate incoming data for todos. Issues: meta - meta is not JSON: {"a": error

Control: the same script without the bad insert prints 2. settled, 3. true ready false 0 and 4. settled ready.

We first hit this in a browser against a real PowerSync service (@powersync/web 2.6.0; Chromium, WebKit, Firefox and Brave under Playwright). The pending write never settled and reloads never became ready in every run: 14 of 14 on rc.1.1, 4 of 4 on rc.1.3.

Expected behavior

  • A pending write settles or rejects. It never hangs.
  • A row that fails deserializationSchema is either reported and skipped (its tracking record deleted, the rest of the batch committed), or the failure is surfaced through the Collection with reportError() / fail(), so health or status shows it. Today status stays ready and health shows nothing while the table stops applying changes.
  • A reload with the bad row in place either becomes ready with the row reported, or fails with a visible error. Today the second preload rejects and the collection stays in error.

Fix pointer: the catch at the end of flushDiffRecordsWithContext. Deserializing each record under its own try/catch, and skipping and reporting the bad one, lets the DELETE, commit() and resolvePendingFor run. The alternative is to reject the pending operations and call fail() / reportError() there.

Workaround we use: a deserializationSchema that never throws. The transform catches its own error and returns a marker value, and the app counts the marked row and leaves it out of the UI. With that, the same browser histories pass every run: the bad row shows as a problem, every write settles, and reloads become ready. It gives up onDeserializationError for those columns.

Desktop

  • OS: macOS
  • Browser: Chromium, WebKit, Firefox, Brave (Playwright), plus the Node repro above
  • Version: @tanstack/powersync-db-collection 1.0.0-rc.1.3 (also 1.0.0-rc.1.1)

Additional context

Related:

  • #2168 (reportError / fail after ready).
  • #2118: no conformance law covers a delivery that the adapter's own schema rejects. The PowerSync driver declares no delivery-error capability and no deserializationSchema, so the oracle never generates this case.
  • #1647: a different cause with the same shape, a pending operation that resolvePendingFor never sees.
Lingua principale
TypeScript
Stelle
4k
Fork
277
Merge medio
1g 11h
PR unite (30g)
148

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di TanStack/db

Tutte le issue di TanStack/db

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.