Request: SQLite IN-list chunking still exceeds the statement-wide parameter limit

Geschlossen
  • Stimmt etwas nicht?
#1,993 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Anfängerfreundlichkeit
45/100
Issue-Typ
Bug
Klarheit
Größtenteils klar
Aktivitätsstatus
Aktiv
Tech-Stack
sqlite, typescript
Bereich
database

Rechercherichtung

Start by tracing the SQLite persistence compiler from adapter.loadSubset and reproduce the issue with the Node SQLite harness using a variable limit of 999. Inspect how composed predicates contribute bound parameters, including index expressions and partial-index predicates. Done means each executed statement stays within SQLite’s parameter limit while preserving typed comparisons, bigint precision, and literal arrays in index definitions.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

Problem

The SQLite persistence compiler splits large IN arrays into smaller clauses joined with OR, but keeps all their bound parameters in one SQL statement.

SQLite's bound-parameter limit applies to the entire statement. Splitting the clauses therefore does not prevent the limit from being exceeded.

Multiple smaller IN predicates can also exceed the limit when composed into one query.

Versions

  • @tanstack/db-sqlite-persistence-core: 0.4.0
  • @tanstack/browser-db-sqlite-persistence: 0.2.25
  • @tanstack/db: 0.11.0

The same compilation strategy is present in upstream main inspected on October 2, 2026.

Reproduction

Use a SQLite-backed adapter with the connection's variable limit set to 999. For example, our Node SQLite harness uses:

new DatabaseSync(':memory:', {
  limits: { variableNumber: 999 },
})

After initializing a collection, request:

import { IR } from '@tanstack/db'

const ids = Array.from({ length: 1_000 }, (_, i) => `issue-${i}`)

await adapter.loadSubset(collectionId, {
  where: new IR.Func('or', [
    new IR.Func('in', [
      new IR.PropRef(['issueId']),
      new IR.Value(ids),
    ]),
    new IR.Func('in', [
      new IR.PropRef(['relatedIssueId']),
      new IR.Value(ids),
    ]),
  ]),
})

Source inspection shows that this produces approximately 2,000 list parameters in one statement, despite the internal chunking. That exceeds the configured limit.

Expected behavior

Bound the total number of parameters in each executed statement, including parameters from composed predicates.

Downstream workaround

We bind each list as one JSON array and compile runtime predicates as:

field IN (SELECT value FROM json_each(?))

An alternative implementation would also be welcome. Important compatibility requirements include preserving typed comparisons and signed 64-bit bigint precision.

Index expressions and partial-index predicates need separate handling: SQLite does not permit these subqueries in index definitions.

Existing downstream tests cover 50,582 IDs across two fields, bigint values beyond JavaScript's safe-integer range, and literal arrays in index definitions. They were not rerun for this source-inspection report.

#810 concerns Electric HTTP request-size limits; this report concerns local SQLite statement parameters.

Vorherrschende Sprache
TypeScript
Sterne
4k
Forks
277
Ø Merge
1 T. 7 Std.
Gemergte PRs (30 T.)
172

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus TanStack/db

Alle Issues in TanStack/db

Ähnliche Issues

Weitere Issues zu TypeScript

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.