setWindow publishes the new page before its include demand settles (on-demand materialize / toArray)

Ouverte
  • Quelque chose ne va pas ?
#2,326 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 1 jour

Personne n'a encore pris cette issue.

Évaluation

Difficulté
4/5
Temps estimé
3-5 jours
Accessibilité débutants
30/100
Type d'issue
Bug
Clarté
Plutôt claire
Activité
Active
Stack technique
typescript
Domaine
databases

Piste de recherche

Start by running the repro script from the issue (bun repro.ts) to confirm the early publish, then read the Atomic window publication and Pending demand sections of packages/db/src/query/live/ARCHITECTURE.md together with the decision in #2131. The question to settle is whether include demand created by a window move should hold publication, which needs a maintainer ruling first. Done means either a fix where the window publishes once with its includes, or a documented and exposed pending-move state, with tests covering the setWindow path.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

  • I've validated the bug against the latest version of DB packages (@tanstack/[email protected]; main has only a docs commit since)

Describe the bug

When a windowed live query has an include over an on-demand source, setWindow() publishes the new page before its include demand settles. The new parent rows appear with empty includes (undefined for materialize(findOne()), [] for toArray()), and the children fill in on a later publication.

The initial load does not do this. preload() waits for the include demand, and the first publication already has every child. The setWindow() promise also waits correctly: it settles only after the children load. Only the publication during the move is early.

This seems to go against Atomic window publication ("the public Collection publishes the completed window once") and the decision in #2131 that an on-demand window "keeps publishing its last complete snapshot" while a window move is in flight. It may instead be intended under Pending demand does not hide the parent row. I couldn't find a rule for which of these applies to include demand that a window move creates.

To Reproduce

Standalone script, no framework: eager parents, so the window move needs no async load, and on-demand children whose loadSubset answers after 200 ms. Run with bun repro.ts.

repro.ts 
// bun repro.ts
import {
  BasicIndex,
  createCollection,
  createLiveQueryCollection,
  eq,
  extractSimpleComparisons,
  localOnlyCollectionOptions,
  materialize,
} from '@tanstack/db'

type Parent = { id: number }
type Child = { id: number; parentId: number; name: string }

const CHILD_LOAD_MS = 200
const indexed = { autoIndex: 'eager', defaultIndexType: BasicIndex } as const
const start = performance.now()
const log = (message: string) => console.log(`${(performance.now() - start).toFixed(0).padStart(4)} ms  ${message}`)

// Eager parents, so the window move itself needs no async load
const parents = createCollection(
  localOnlyCollectionOptions({
    ...indexed,
    getKey: (p: Parent) => p.id,
    initialData: Array.from({ length: 9 }, (_, i) => ({ id: i + 1 })),
  }),
)

// On-demand children, served by a backend that answers after CHILD_LOAD_MS
const backend: Array<Child> = Array.from({ length: 9 }, (_, i) => ({
  id: 101 + i,
  parentId: i + 1,
  name: `child-of-${i + 1}`,
}))

const children = createCollection<Child, number>({
  ...indexed,
  getKey: (c) => c.id,
  syncMode: 'on-demand',
  sync: {
    sync: ({ begin, write, commit, markReady }) => {
      markReady()
      return {
        loadSubset: async (options) => {
          const parentIds = new Set<number>()
          for (const f of extractSimpleComparisons(options.where)) {
            if (f.field.join('.') !== 'parentId') continue
            for (const v of f.operator === 'in' ? (f.value as Array<number>) : [f.value as number]) parentIds.add(v)
          }
          log(`  children.loadSubset parentIds=[${[...parentIds]}]`)
          await new Promise((r) => setTimeout(r, CHILD_LOAD_MS))
          begin()
          for (const c of backend.filter((c) => parentIds.has(c.parentId))) write({ type: 'insert', value: c })
          commit()
          log('  children.loadSubset resolved')
        },
      }
    },
  },
})

const page = createLiveQueryCollection({
  query: (q) =>
    q
      .from({ p: parents })
      .orderBy(({ p }) => p.id)
      .limit(3)
      .select(({ p }) => ({
        id: p.id,
        child: materialize(
          q
            .from({ c: children })
            .where(({ c }) => eq(c.parentId, p.id))
            .findOne(),
        ),
      })),
})

const show = () => `[${page.toArray.map((r) => `${r.id}:${r.child?.name ?? 'MISSING'}`).join(', ')}]`

page.subscribeChanges(() => log(`publish ${show()}  isLoadingSubset=${page.isLoadingSubset}`))

await page.preload()
log(`preload settled ${show()}`)

log('setWindow({ offset: 3, limit: 3 })')
const settlement = page.utils.setWindow({ offset: 3, limit: 3 })
if (settlement !== true) await settlement
log(`setWindow settled ${show()}`)

Output:

  26 ms    children.loadSubset parentIds=[1,2,3]
 230 ms  publish [1:child-of-1, 2:child-of-2, 3:child-of-3]  isLoadingSubset=false
 231 ms    children.loadSubset resolved
 231 ms  publish [1:child-of-1, 2:child-of-2, 3:child-of-3]  isLoadingSubset=false
 231 ms  preload settled [1:child-of-1, 2:child-of-2, 3:child-of-3]
 231 ms  setWindow({ offset: 3, limit: 3 })
 234 ms    children.loadSubset parentIds=[4,5,6]
 235 ms  publish [4:MISSING, 5:MISSING, 6:MISSING]  isLoadingSubset=true    <-- page published before its includes
 436 ms  publish [4:child-of-4, 5:child-of-5, 6:child-of-6]  isLoadingSubset=false
 436 ms    children.loadSubset resolved
 438 ms  setWindow settled [4:child-of-4, 5:child-of-5, 6:child-of-6]

Replacing materialize(...findOne()) with toArray(...) gives the same sequence, with [] instead of undefined.

Expected behavior

The same as the initial load: while the move is pending, keep publishing the last complete window ([1, 2, 3] with their children). Then publish [4, 5, 6] once, with their children, when the setWindow() promise settles.

If publishing the parents early is intended, it would help to document that for window moves, and for the hooks to expose that a move is pending. Today useLiveQuery's isLoading / isReady only cover the initial load, and isLoadingSubset isn't returned. An app can only hide the partial page by awaiting the setWindow() promise itself.

Real-world impact

A paginated table with numbered pages, over Electric on-demand collections. The query has an orderBy and materialize includes for the related rows, and the page is moved with setWindow (the workaround described in #2291). On every page change, the cells built from includes (reference, client, delivery date, a status computed from a related row) show their empty fallback for about 900 ms. Then they switch to the real values. A status derived from the related row can briefly be wrong, not just empty.

Environment

Additional context

  • #2291 / #2316: page moves through useLiveQuery. #2316 lists "new include-publication semantics" as out of scope, so this behavior would carry over to adapter-owned page windows.
  • #2131: the policy decision to hold the last complete snapshot while a window move is in flight.
  • #2156: includes are routed only from the rows the window shows. That is why a move creates new child demand at all.
Langage dominant
TypeScript
Étoiles
4k
Forks
277
Merge moyen
1 j 7 h
PR mergées (30 j)
172

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de TanStack/db

Toutes les issues de TanStack/db

Issues similaires

Plus d'issues TypeScript

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.