sqlite-persistence: deleted rows keep their full value in the tombstone table forever (local-only collections)

Abierto
#2,344 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
35/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
node.js, sqlite, typescript

Línea de trabajo

Start with the tombstone INSERT that uses value = excluded.value in sqlite-core-adapter.js, and the mutation.original read in persisted.js that supplies the value. Reproduce with the Node snippet in the issue and check the sqlite_master tables for the value. Done means deleted rows keep only key and row version in tombstones, or a supported way to clear a local-only collection exists; this touches the sync and pruning logic, so design input from maintainers is needed first.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Versions: @tanstack/db, @tanstack/db-sqlite-persistence-core, @tanstack/node-db-sqlite-persistence, @tanstack/browser-db-sqlite-persistence and @tanstack/expo-db-sqlite-persistence 1.0.0-rc.1.2.

What happens

In a persisted collection with no sync (persistedCollectionOptions without sync, writes through utils.acceptMutations), deleting a row writes the row's whole value to the collection's tombstone table (t_<hash>). Nothing removes it later: a tombstone goes only when the same key is written again, on a truncate, or on a schema reset. A local-only collection can't truncate. The value also stays in applied_tx.replay_json until that table is pruned (1000 rows / 24 h by default).

So after delete(), the row is gone from the collection but still readable in the SQLite file (OPFS on the web, the app's files on phones).

Repro (Node)

import Database from 'better-sqlite3'
import { createCollection, createTransaction } from '@tanstack/db'
import { persistedCollectionOptions } from '@tanstack/db-sqlite-persistence-core'
import { createNodeSQLitePersistence } from '@tanstack/node-db-sqlite-persistence'

const database = new Database(':memory:')
const todos = createCollection(
  persistedCollectionOptions<{ id: string; text: string }, string>({
    id: 'todos',
    getKey: (todo) => todo.id,
    persistence: createNodeSQLitePersistence({ database }),
  }),
)
await todos.preload()
const write = async (change: () => void) => {
  const tx = createTransaction({
    autoCommit: false,
    mutationFn: async ({ transaction }) => {
      await todos.utils.acceptMutations(transaction)
    },
  })
  tx.mutate(change)
  await tx.commit()
}
await write(() => todos.insert({ id: '1', text: 'private note' }))
await write(() => todos.delete('1'))

console.log(todos.size) // 0
for (const { name } of database.prepare(`SELECT name FROM sqlite_master WHERE type = 'table'`).all()) {
  const rows = database.prepare(`SELECT * FROM "${name}"`).all()
  if (JSON.stringify(rows).includes('private note')) console.log(name)
}
// applied_tx    (replay_json has the inserted value)
// t_bicmgyk_5   {"key":"s:1","value":"{\"id\":\"1\",\"text\":\"private note\"}","row_version":2,"deleted_at":"…"}

Why it matters

  • Privacy on shared devices. We clear a person's data at Sign Out by deleting their rows. With this behaviour the next person on the device (or a script on the site) can still read them from the file. We now delete the whole SQLite file at Sign Out instead.
  • Growth. Every row that's written once and deleted (an outbox entry, say) leaves a tombstone for good, so the file grows without limit.

What would help

Any of these:

  • Store only the key (and row version) in a tombstone, not the value.
  • Prune tombstones (by count or age, like appliedTxPruneMaxRows / appliedTxPruneMaxAgeSeconds) once no tab needs them for catch-up.
  • A supported way to empty a local-only collection completely, tombstones and applied_tx included (a local truncate), or to delete a collection's tables.

The relevant code is in sqlite-core-adapter.js: the tombstone INSERT … value = excluded.value for deletes, and the only DELETE FROM tombstone paths (a truncate, a re-write of the same key, a schema reset). The value comes from mutation.original in persisted.js.

Lenguaje dominante
TypeScript
Estrellas
4k
Forks
286
Merge medio
1 d 8 h
PR fusionados (30 d)
105

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de TanStack/db

Todos los issues de TanStack/db

Issues similares

Más issues de TypeScript

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.