Hacktoberfest 2026 : les issues que les mainteneurs ont marquées pour octobre, ouvertes et accessibles aux débutants. Parcourir les issues Hacktoberfest

On-demand ordered windows wait on a boundary tie request even when ordering by the collection key

Ouverte
  • Quelque chose ne va pas ?
#2,166 1 commentaire 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 1 jour

Personne n'a encore pris cette issue.

Évaluation

Difficulté
4/5
Temps estimé
3-5 jours
Accessibilité débutants
18/100
Type d'issue
Bug
Clarté
Plutôt claire
Activité
Active
Stack technique
typescript
Domaine
data-engineering

Piste de recherche

Start in packages/db/src/query/live/ordered-source-loader.ts, around the sequencing at L745-L752 and the boundary request at L880, then look at buildCursorCurrent in packages/db/src/utils/cursor.ts. The repro.mjs script in the issue shows the two extra loadSubset calls per window step. Done means one loadSubset per step when the leading order term is unique, without changing behaviour for tied keys.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

Describe the bug

An on-demand live query ordered by the collection key (getKey: row => row.id and .orderBy(({ item }) => item.id)) sends an extra loadSubset after every ordered page. It has where: <query where> AND id = <last loaded id>, with no orderBy and no limit. id is unique, so this request can only return the row that was just loaded.

Since 0.9.0 (#1797), the ordered source loader waits for it: ordered → boundary → more. Each window step therefore costs two sequential round trips instead of one, and setWindow() resolves only after the boundary request returns. In 0.6.14 a window step was a single cursor request.

With the Electric adapter, the following cursor request also sends cursor.whereCurrent (id = <last loaded id>) alongside whereFrom. Each step therefore makes two requests whose only result is a row the collection already holds.

Reproduction

A plain on-demand collection with a logging loadSubset and 50 ms of simulated latency. No adapter is involved. The script is in the details below.

@tanstack/[email protected]:

   4-  57 ms  page      limit=5    -> 5 rows
  57- 109 ms  unordered limit=null -> 1 rows   (list = 'a' AND id = 'row-004')
 413- 466 ms  cursor    limit=5    -> 6 rows
 466- 518 ms  unordered limit=null -> 1 rows   (list = 'a' AND id = 'row-009')
setWindow({ limit: 10 }) took 107 ms

@tanstack/[email protected], same script:

   3-  60 ms  page      limit=5    -> 5 rows
 363- 416 ms  cursor    limit=5    -> 6 rows
setWindow({ limit: 10 }) took 55 ms
repro.mjs (run with node repro.mjs next to an installed @tanstack/db) 
import { BTreeIndex, createCollection, createLiveQueryCollection, eq, parseWhereExpression } from '@tanstack/db'

const LATENCY_MS = 50
const serverRows = Array.from({ length: 20 }, (_, i) => ({ id: `row-${String(i).padStart(3, '0')}`, list: 'a' }))

const toPredicate = where =>
  where === undefined
    ? () => true
    : parseWhereExpression(where, {
        handlers: {
          eq: ([f], v) => r => r[f] === v,
          gt: ([f], v) => r => r[f] > v,
          isNull: ([f]) => r => r[f] === null,
          isUndefined: ([f]) => r => r[f] === undefined,
          and: (...ps) => r => ps.every(p => p(r)),
          or: (...ps) => r => ps.some(p => p(r)),
        },
      })
const select = (where, limit) =>
  serverRows.filter(toPredicate(where)).sort((a, b) => (a.id < b.id ? -1 : 1)).slice(0, limit)
const and2 = (a, b) => ({ type: 'func', name: 'and', args: [a, b] })

const t0 = Date.now()
const calls = []
const items = createCollection({
  id: 'items',
  getKey: row => row.id,
  syncMode: 'on-demand',
  defaultIndexType: BTreeIndex,
  defaultStringCollation: { stringSort: 'lexical' },
  sync: {
    sync: ({ begin, write, commit, markReady }) => {
      markReady()
      return {
        loadSubset: async options => {
          const call = {
            start: Date.now() - t0,
            kind: options.cursor ? 'cursor' : options.orderBy ? 'page' : 'unordered',
            limit: options.limit ?? null,
          }
          calls.push(call)
          await new Promise(resolve => setTimeout(resolve, LATENCY_MS))
          // Like the Electric adapter: a cursor request loads whereCurrent ties plus whereFrom rows.
          const rows = options.cursor
            ? [
                ...select(and2(options.where, options.cursor.whereCurrent)),
                ...select(and2(options.where, options.cursor.whereFrom), options.limit),
              ]
            : select(options.where, options.limit)
          begin()
          for (const row of rows) write({ type: 'insert', value: row })
          commit()
          call.end = Date.now() - t0
          call.rows = rows.length
        },
      }
    },
  },
})
items.createIndex(row => row.id, {
  indexType: BTreeIndex,
  options: { compareOptions: { ...items.compareOptions, direction: 'asc', nulls: 'last' } },
})

const windowed = createLiveQueryCollection(q =>
  q
    .from({ item: items })
    .where(({ item }) => eq(item.list, 'a'))
    .orderBy(({ item }) => item.id, { direction: 'asc', nulls: 'last' })
    .limit(5)
)
await windowed.preload()
await new Promise(resolve => setTimeout(resolve, 300))

const calledAt = Date.now() - t0
await windowed.utils.setWindow({ offset: 0, limit: 10 })
const resolvedAt = Date.now() - t0
await new Promise(resolve => setTimeout(resolve, 300))

for (const call of calls) {
  console.log(`${String(call.start).padStart(4)}-${String(call.end).padStart(4)} ms  ${call.kind.padEnd(9)} limit=${String(call.limit).padEnd(4)} -> ${call.rows} rows`)
}
console.log(`setWindow({ limit: 10 }) took ${resolvedAt - calledAt} ms`)
Impact

We backfill a large on-demand Electric collection in 1,000-row windows by calling setWindow repeatedly, ordered by id. In our E2E benchmark (280 ms request latency, medians of 3 runs), the only change between the two runs was the upgrade from 0.6.14 to 0.13.0:

0.6.14 0.13.0
30k rows: sequential round trips 31 63
30k rows: time to load all rows 12.9 s 23.6 s
70k rows plus dependent output queries: time to load all rows 73 s 114 s

In production, every 1,000-row page also showed both id = <last id> requests, each returning the one row already loaded.

Expected behavior

When the leading order term is unique, a window step needs one loadSubset. No row can tie with the boundary, so neither the boundary request nor cursor.whereCurrent can add anything.

Possible approaches

Core can't infer from a getKey function that id is unique, so this probably needs an explicit signal. Ideas:

  1. An opt-in that marks the leading order term as unique, for example a collection option naming the key path, or unique: true on the order-by options. With it:
    • loadBoundary would skip the tie request;
    • adapters would leave out cursor.whereCurrent, perhaps via a flag on LoadSubsetOptions/CursorExpressions.
  2. Independent of uniqueness: don't hold the next window step or setWindow() resolution on the boundary request, so it costs no added latency. That was the 0.6.14 behavior, where the tie travelled with the cursor request.
Code references
Versions
  • @tanstack/db 0.13.0; the same code is on main / 0.14.0.
  • @tanstack/electric-db-collection 0.5.9.
  • @tanstack/react-db 0.5.8.
  • Node 24; also seen in Chrome in production.
Langage dominant
TypeScript
Étoiles
4k
Forks
277
Merge moyen
21 h 21 min
PR mergées (30 j)
218

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de TanStack/db

Toutes les issues de TanStack/db

Issues similaires

Plus d'issues TypeScript

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.