Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

On-demand ordered windows wait on a boundary tie request even when ordering by the collection key

Aperta
  • Qualcosa non va?
#2,166 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
18/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
typescript
Ambito
data-engineering

Direzione di ricerca

Start in packages/db/src/query/live/ordered-source-loader.ts, around the sequencing at L745-L752 and the boundary request at L880, then look at buildCursorCurrent in packages/db/src/utils/cursor.ts. The repro.mjs script in the issue shows the two extra loadSubset calls per window step. Done means one loadSubset per step when the leading order term is unique, without changing behaviour for tied keys.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Describe the bug

An on-demand live query ordered by the collection key (getKey: row => row.id and .orderBy(({ item }) => item.id)) sends an extra loadSubset after every ordered page. It has where: <query where> AND id = <last loaded id>, with no orderBy and no limit. id is unique, so this request can only return the row that was just loaded.

Since 0.9.0 (#1797), the ordered source loader waits for it: ordered → boundary → more. Each window step therefore costs two sequential round trips instead of one, and setWindow() resolves only after the boundary request returns. In 0.6.14 a window step was a single cursor request.

With the Electric adapter, the following cursor request also sends cursor.whereCurrent (id = <last loaded id>) alongside whereFrom. Each step therefore makes two requests whose only result is a row the collection already holds.

Reproduction

A plain on-demand collection with a logging loadSubset and 50 ms of simulated latency. No adapter is involved. The script is in the details below.

@tanstack/[email protected]:

   4-  57 ms  page      limit=5    -> 5 rows
  57- 109 ms  unordered limit=null -> 1 rows   (list = 'a' AND id = 'row-004')
 413- 466 ms  cursor    limit=5    -> 6 rows
 466- 518 ms  unordered limit=null -> 1 rows   (list = 'a' AND id = 'row-009')
setWindow({ limit: 10 }) took 107 ms

@tanstack/[email protected], same script:

   3-  60 ms  page      limit=5    -> 5 rows
 363- 416 ms  cursor    limit=5    -> 6 rows
setWindow({ limit: 10 }) took 55 ms
repro.mjs (run with node repro.mjs next to an installed @tanstack/db) 
import { BTreeIndex, createCollection, createLiveQueryCollection, eq, parseWhereExpression } from '@tanstack/db'

const LATENCY_MS = 50
const serverRows = Array.from({ length: 20 }, (_, i) => ({ id: `row-${String(i).padStart(3, '0')}`, list: 'a' }))

const toPredicate = where =>
  where === undefined
    ? () => true
    : parseWhereExpression(where, {
        handlers: {
          eq: ([f], v) => r => r[f] === v,
          gt: ([f], v) => r => r[f] > v,
          isNull: ([f]) => r => r[f] === null,
          isUndefined: ([f]) => r => r[f] === undefined,
          and: (...ps) => r => ps.every(p => p(r)),
          or: (...ps) => r => ps.some(p => p(r)),
        },
      })
const select = (where, limit) =>
  serverRows.filter(toPredicate(where)).sort((a, b) => (a.id < b.id ? -1 : 1)).slice(0, limit)
const and2 = (a, b) => ({ type: 'func', name: 'and', args: [a, b] })

const t0 = Date.now()
const calls = []
const items = createCollection({
  id: 'items',
  getKey: row => row.id,
  syncMode: 'on-demand',
  defaultIndexType: BTreeIndex,
  defaultStringCollation: { stringSort: 'lexical' },
  sync: {
    sync: ({ begin, write, commit, markReady }) => {
      markReady()
      return {
        loadSubset: async options => {
          const call = {
            start: Date.now() - t0,
            kind: options.cursor ? 'cursor' : options.orderBy ? 'page' : 'unordered',
            limit: options.limit ?? null,
          }
          calls.push(call)
          await new Promise(resolve => setTimeout(resolve, LATENCY_MS))
          // Like the Electric adapter: a cursor request loads whereCurrent ties plus whereFrom rows.
          const rows = options.cursor
            ? [
                ...select(and2(options.where, options.cursor.whereCurrent)),
                ...select(and2(options.where, options.cursor.whereFrom), options.limit),
              ]
            : select(options.where, options.limit)
          begin()
          for (const row of rows) write({ type: 'insert', value: row })
          commit()
          call.end = Date.now() - t0
          call.rows = rows.length
        },
      }
    },
  },
})
items.createIndex(row => row.id, {
  indexType: BTreeIndex,
  options: { compareOptions: { ...items.compareOptions, direction: 'asc', nulls: 'last' } },
})

const windowed = createLiveQueryCollection(q =>
  q
    .from({ item: items })
    .where(({ item }) => eq(item.list, 'a'))
    .orderBy(({ item }) => item.id, { direction: 'asc', nulls: 'last' })
    .limit(5)
)
await windowed.preload()
await new Promise(resolve => setTimeout(resolve, 300))

const calledAt = Date.now() - t0
await windowed.utils.setWindow({ offset: 0, limit: 10 })
const resolvedAt = Date.now() - t0
await new Promise(resolve => setTimeout(resolve, 300))

for (const call of calls) {
  console.log(`${String(call.start).padStart(4)}-${String(call.end).padStart(4)} ms  ${call.kind.padEnd(9)} limit=${String(call.limit).padEnd(4)} -> ${call.rows} rows`)
}
console.log(`setWindow({ limit: 10 }) took ${resolvedAt - calledAt} ms`)
Impact

We backfill a large on-demand Electric collection in 1,000-row windows by calling setWindow repeatedly, ordered by id. In our E2E benchmark (280 ms request latency, medians of 3 runs), the only change between the two runs was the upgrade from 0.6.14 to 0.13.0:

0.6.14 0.13.0
30k rows: sequential round trips 31 63
30k rows: time to load all rows 12.9 s 23.6 s
70k rows plus dependent output queries: time to load all rows 73 s 114 s

In production, every 1,000-row page also showed both id = <last id> requests, each returning the one row already loaded.

Expected behavior

When the leading order term is unique, a window step needs one loadSubset. No row can tie with the boundary, so neither the boundary request nor cursor.whereCurrent can add anything.

Possible approaches

Core can't infer from a getKey function that id is unique, so this probably needs an explicit signal. Ideas:

  1. An opt-in that marks the leading order term as unique, for example a collection option naming the key path, or unique: true on the order-by options. With it:
    • loadBoundary would skip the tie request;
    • adapters would leave out cursor.whereCurrent, perhaps via a flag on LoadSubsetOptions/CursorExpressions.
  2. Independent of uniqueness: don't hold the next window step or setWindow() resolution on the boundary request, so it costs no added latency. That was the 0.6.14 behavior, where the tie travelled with the cursor request.
Code references
Versions
  • @tanstack/db 0.13.0; the same code is on main / 0.14.0.
  • @tanstack/electric-db-collection 0.5.9.
  • @tanstack/react-db 0.5.8.
  • Node 24; also seen in Chrome in production.
Lingua principale
TypeScript
Stelle
4k
Fork
277
Merge medio
17h 40m
PR unite (30g)
223

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di TanStack/db

Tutte le issue di TanStack/db

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.