setWindow publishes the new page before its include demand settles (on-demand materialize / toArray)

Offen
  • Stimmt etwas nicht?
#2,326 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Anfängerfreundlichkeit
30/100
Issue-Typ
Bug
Klarheit
Größtenteils klar
Aktivitätsstatus
Aktiv
Tech-Stack
typescript
Bereich
databases

Rechercherichtung

Start by running the repro script from the issue (bun repro.ts) to confirm the early publish, then read the Atomic window publication and Pending demand sections of packages/db/src/query/live/ARCHITECTURE.md together with the decision in #2131. The question to settle is whether include demand created by a window move should hold publication, which needs a maintainer ruling first. Done means either a fix where the window publishes once with its includes, or a documented and exposed pending-move state, with tests covering the setWindow path.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

  • I've validated the bug against the latest version of DB packages (@tanstack/[email protected]; main has only a docs commit since)

Describe the bug

When a windowed live query has an include over an on-demand source, setWindow() publishes the new page before its include demand settles. The new parent rows appear with empty includes (undefined for materialize(findOne()), [] for toArray()), and the children fill in on a later publication.

The initial load does not do this. preload() waits for the include demand, and the first publication already has every child. The setWindow() promise also waits correctly: it settles only after the children load. Only the publication during the move is early.

This seems to go against Atomic window publication ("the public Collection publishes the completed window once") and the decision in #2131 that an on-demand window "keeps publishing its last complete snapshot" while a window move is in flight. It may instead be intended under Pending demand does not hide the parent row. I couldn't find a rule for which of these applies to include demand that a window move creates.

To Reproduce

Standalone script, no framework: eager parents, so the window move needs no async load, and on-demand children whose loadSubset answers after 200 ms. Run with bun repro.ts.

repro.ts 
// bun repro.ts
import {
  BasicIndex,
  createCollection,
  createLiveQueryCollection,
  eq,
  extractSimpleComparisons,
  localOnlyCollectionOptions,
  materialize,
} from '@tanstack/db'

type Parent = { id: number }
type Child = { id: number; parentId: number; name: string }

const CHILD_LOAD_MS = 200
const indexed = { autoIndex: 'eager', defaultIndexType: BasicIndex } as const
const start = performance.now()
const log = (message: string) => console.log(`${(performance.now() - start).toFixed(0).padStart(4)} ms  ${message}`)

// Eager parents, so the window move itself needs no async load
const parents = createCollection(
  localOnlyCollectionOptions({
    ...indexed,
    getKey: (p: Parent) => p.id,
    initialData: Array.from({ length: 9 }, (_, i) => ({ id: i + 1 })),
  }),
)

// On-demand children, served by a backend that answers after CHILD_LOAD_MS
const backend: Array<Child> = Array.from({ length: 9 }, (_, i) => ({
  id: 101 + i,
  parentId: i + 1,
  name: `child-of-${i + 1}`,
}))

const children = createCollection<Child, number>({
  ...indexed,
  getKey: (c) => c.id,
  syncMode: 'on-demand',
  sync: {
    sync: ({ begin, write, commit, markReady }) => {
      markReady()
      return {
        loadSubset: async (options) => {
          const parentIds = new Set<number>()
          for (const f of extractSimpleComparisons(options.where)) {
            if (f.field.join('.') !== 'parentId') continue
            for (const v of f.operator === 'in' ? (f.value as Array<number>) : [f.value as number]) parentIds.add(v)
          }
          log(`  children.loadSubset parentIds=[${[...parentIds]}]`)
          await new Promise((r) => setTimeout(r, CHILD_LOAD_MS))
          begin()
          for (const c of backend.filter((c) => parentIds.has(c.parentId))) write({ type: 'insert', value: c })
          commit()
          log('  children.loadSubset resolved')
        },
      }
    },
  },
})

const page = createLiveQueryCollection({
  query: (q) =>
    q
      .from({ p: parents })
      .orderBy(({ p }) => p.id)
      .limit(3)
      .select(({ p }) => ({
        id: p.id,
        child: materialize(
          q
            .from({ c: children })
            .where(({ c }) => eq(c.parentId, p.id))
            .findOne(),
        ),
      })),
})

const show = () => `[${page.toArray.map((r) => `${r.id}:${r.child?.name ?? 'MISSING'}`).join(', ')}]`

page.subscribeChanges(() => log(`publish ${show()}  isLoadingSubset=${page.isLoadingSubset}`))

await page.preload()
log(`preload settled ${show()}`)

log('setWindow({ offset: 3, limit: 3 })')
const settlement = page.utils.setWindow({ offset: 3, limit: 3 })
if (settlement !== true) await settlement
log(`setWindow settled ${show()}`)

Output:

  26 ms    children.loadSubset parentIds=[1,2,3]
 230 ms  publish [1:child-of-1, 2:child-of-2, 3:child-of-3]  isLoadingSubset=false
 231 ms    children.loadSubset resolved
 231 ms  publish [1:child-of-1, 2:child-of-2, 3:child-of-3]  isLoadingSubset=false
 231 ms  preload settled [1:child-of-1, 2:child-of-2, 3:child-of-3]
 231 ms  setWindow({ offset: 3, limit: 3 })
 234 ms    children.loadSubset parentIds=[4,5,6]
 235 ms  publish [4:MISSING, 5:MISSING, 6:MISSING]  isLoadingSubset=true    <-- page published before its includes
 436 ms  publish [4:child-of-4, 5:child-of-5, 6:child-of-6]  isLoadingSubset=false
 436 ms    children.loadSubset resolved
 438 ms  setWindow settled [4:child-of-4, 5:child-of-5, 6:child-of-6]

Replacing materialize(...findOne()) with toArray(...) gives the same sequence, with [] instead of undefined.

Expected behavior

The same as the initial load: while the move is pending, keep publishing the last complete window ([1, 2, 3] with their children). Then publish [4, 5, 6] once, with their children, when the setWindow() promise settles.

If publishing the parents early is intended, it would help to document that for window moves, and for the hooks to expose that a move is pending. Today useLiveQuery's isLoading / isReady only cover the initial load, and isLoadingSubset isn't returned. An app can only hide the partial page by awaiting the setWindow() promise itself.

Real-world impact

A paginated table with numbered pages, over Electric on-demand collections. The query has an orderBy and materialize includes for the related rows, and the page is moved with setWindow (the workaround described in #2291). On every page change, the cells built from includes (reference, client, delivery date, a status computed from a related row) show their empty fallback for about 900 ms. Then they switch to the real values. A status derived from the related row can briefly be wrong, not just empty.

Environment

Additional context

  • #2291 / #2316: page moves through useLiveQuery. #2316 lists "new include-publication semantics" as out of scope, so this behavior would carry over to adapter-owned page windows.
  • #2131: the policy decision to hold the last complete snapshot while a window move is in flight.
  • #2156: includes are routed only from the rows the window shows. That is why a move creates new child demand at all.
Vorherrschende Sprache
TypeScript
Sterne
4k
Forks
277
Ø Merge
1 T. 7 Std.
Gemergte PRs (30 T.)
172

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus TanStack/db

Alle Issues in TanStack/db

Ähnliche Issues

Weitere Issues zu TypeScript

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.