Hacktoberfest 2026: die Issues, die Maintainer für den Oktober markiert haben – offen und einsteigerfreundlich. Hacktoberfest-Issues durchsuchen

On-demand ordered windows wait on a boundary tie request even when ordering by the collection key

Offen
  • Stimmt etwas nicht?
#2,166 1 Kommentar 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Anfängerfreundlichkeit
18/100
Issue-Typ
Bug
Klarheit
Größtenteils klar
Aktivitätsstatus
Aktiv
Tech-Stack
typescript
Bereich
data-engineering

Rechercherichtung

Start in packages/db/src/query/live/ordered-source-loader.ts, around the sequencing at L745-L752 and the boundary request at L880, then look at buildCursorCurrent in packages/db/src/utils/cursor.ts. The repro.mjs script in the issue shows the two extra loadSubset calls per window step. Done means one loadSubset per step when the leading order term is unique, without changing behaviour for tied keys.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

Describe the bug

An on-demand live query ordered by the collection key (getKey: row => row.id and .orderBy(({ item }) => item.id)) sends an extra loadSubset after every ordered page. It has where: <query where> AND id = <last loaded id>, with no orderBy and no limit. id is unique, so this request can only return the row that was just loaded.

Since 0.9.0 (#1797), the ordered source loader waits for it: ordered → boundary → more. Each window step therefore costs two sequential round trips instead of one, and setWindow() resolves only after the boundary request returns. In 0.6.14 a window step was a single cursor request.

With the Electric adapter, the following cursor request also sends cursor.whereCurrent (id = <last loaded id>) alongside whereFrom. Each step therefore makes two requests whose only result is a row the collection already holds.

Reproduction

A plain on-demand collection with a logging loadSubset and 50 ms of simulated latency. No adapter is involved. The script is in the details below.

@tanstack/[email protected]:

   4-  57 ms  page      limit=5    -> 5 rows
  57- 109 ms  unordered limit=null -> 1 rows   (list = 'a' AND id = 'row-004')
 413- 466 ms  cursor    limit=5    -> 6 rows
 466- 518 ms  unordered limit=null -> 1 rows   (list = 'a' AND id = 'row-009')
setWindow({ limit: 10 }) took 107 ms

@tanstack/[email protected], same script:

   3-  60 ms  page      limit=5    -> 5 rows
 363- 416 ms  cursor    limit=5    -> 6 rows
setWindow({ limit: 10 }) took 55 ms
repro.mjs (run with node repro.mjs next to an installed @tanstack/db) 
import { BTreeIndex, createCollection, createLiveQueryCollection, eq, parseWhereExpression } from '@tanstack/db'

const LATENCY_MS = 50
const serverRows = Array.from({ length: 20 }, (_, i) => ({ id: `row-${String(i).padStart(3, '0')}`, list: 'a' }))

const toPredicate = where =>
  where === undefined
    ? () => true
    : parseWhereExpression(where, {
        handlers: {
          eq: ([f], v) => r => r[f] === v,
          gt: ([f], v) => r => r[f] > v,
          isNull: ([f]) => r => r[f] === null,
          isUndefined: ([f]) => r => r[f] === undefined,
          and: (...ps) => r => ps.every(p => p(r)),
          or: (...ps) => r => ps.some(p => p(r)),
        },
      })
const select = (where, limit) =>
  serverRows.filter(toPredicate(where)).sort((a, b) => (a.id < b.id ? -1 : 1)).slice(0, limit)
const and2 = (a, b) => ({ type: 'func', name: 'and', args: [a, b] })

const t0 = Date.now()
const calls = []
const items = createCollection({
  id: 'items',
  getKey: row => row.id,
  syncMode: 'on-demand',
  defaultIndexType: BTreeIndex,
  defaultStringCollation: { stringSort: 'lexical' },
  sync: {
    sync: ({ begin, write, commit, markReady }) => {
      markReady()
      return {
        loadSubset: async options => {
          const call = {
            start: Date.now() - t0,
            kind: options.cursor ? 'cursor' : options.orderBy ? 'page' : 'unordered',
            limit: options.limit ?? null,
          }
          calls.push(call)
          await new Promise(resolve => setTimeout(resolve, LATENCY_MS))
          // Like the Electric adapter: a cursor request loads whereCurrent ties plus whereFrom rows.
          const rows = options.cursor
            ? [
                ...select(and2(options.where, options.cursor.whereCurrent)),
                ...select(and2(options.where, options.cursor.whereFrom), options.limit),
              ]
            : select(options.where, options.limit)
          begin()
          for (const row of rows) write({ type: 'insert', value: row })
          commit()
          call.end = Date.now() - t0
          call.rows = rows.length
        },
      }
    },
  },
})
items.createIndex(row => row.id, {
  indexType: BTreeIndex,
  options: { compareOptions: { ...items.compareOptions, direction: 'asc', nulls: 'last' } },
})

const windowed = createLiveQueryCollection(q =>
  q
    .from({ item: items })
    .where(({ item }) => eq(item.list, 'a'))
    .orderBy(({ item }) => item.id, { direction: 'asc', nulls: 'last' })
    .limit(5)
)
await windowed.preload()
await new Promise(resolve => setTimeout(resolve, 300))

const calledAt = Date.now() - t0
await windowed.utils.setWindow({ offset: 0, limit: 10 })
const resolvedAt = Date.now() - t0
await new Promise(resolve => setTimeout(resolve, 300))

for (const call of calls) {
  console.log(`${String(call.start).padStart(4)}-${String(call.end).padStart(4)} ms  ${call.kind.padEnd(9)} limit=${String(call.limit).padEnd(4)} -> ${call.rows} rows`)
}
console.log(`setWindow({ limit: 10 }) took ${resolvedAt - calledAt} ms`)
Impact

We backfill a large on-demand Electric collection in 1,000-row windows by calling setWindow repeatedly, ordered by id. In our E2E benchmark (280 ms request latency, medians of 3 runs), the only change between the two runs was the upgrade from 0.6.14 to 0.13.0:

0.6.14 0.13.0
30k rows: sequential round trips 31 63
30k rows: time to load all rows 12.9 s 23.6 s
70k rows plus dependent output queries: time to load all rows 73 s 114 s

In production, every 1,000-row page also showed both id = <last id> requests, each returning the one row already loaded.

Expected behavior

When the leading order term is unique, a window step needs one loadSubset. No row can tie with the boundary, so neither the boundary request nor cursor.whereCurrent can add anything.

Possible approaches

Core can't infer from a getKey function that id is unique, so this probably needs an explicit signal. Ideas:

  1. An opt-in that marks the leading order term as unique, for example a collection option naming the key path, or unique: true on the order-by options. With it:
    • loadBoundary would skip the tie request;
    • adapters would leave out cursor.whereCurrent, perhaps via a flag on LoadSubsetOptions/CursorExpressions.
  2. Independent of uniqueness: don't hold the next window step or setWindow() resolution on the boundary request, so it costs no added latency. That was the 0.6.14 behavior, where the tie travelled with the cursor request.
Code references
Versions
  • @tanstack/db 0.13.0; the same code is on main / 0.14.0.
  • @tanstack/electric-db-collection 0.5.9.
  • @tanstack/react-db 0.5.8.
  • Node 24; also seen in Chrome in production.
Vorherrschende Sprache
TypeScript
Sterne
4k
Forks
277
Ø Merge
1 T. 2 Std.
Gemergte PRs (30 T.)
213

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus TanStack/db

Alle Issues in TanStack/db

Ähnliche Issues

Weitere Issues zu TypeScript

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.