setWindow publishes the new page before its include demand settles (on-demand materialize / toArray)

Abierto
  • ¿Algo no está bien?
#2,326 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
30/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
typescript
Área
databases

Línea de trabajo

Start by running the repro script from the issue (bun repro.ts) to confirm the early publish, then read the Atomic window publication and Pending demand sections of packages/db/src/query/live/ARCHITECTURE.md together with the decision in #2131. The question to settle is whether include demand created by a window move should hold publication, which needs a maintainer ruling first. Done means either a fix where the window publishes once with its includes, or a documented and exposed pending-move state, with tests covering the setWindow path.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

  • I've validated the bug against the latest version of DB packages (@tanstack/[email protected]; main has only a docs commit since)

Describe the bug

When a windowed live query has an include over an on-demand source, setWindow() publishes the new page before its include demand settles. The new parent rows appear with empty includes (undefined for materialize(findOne()), [] for toArray()), and the children fill in on a later publication.

The initial load does not do this. preload() waits for the include demand, and the first publication already has every child. The setWindow() promise also waits correctly: it settles only after the children load. Only the publication during the move is early.

This seems to go against Atomic window publication ("the public Collection publishes the completed window once") and the decision in #2131 that an on-demand window "keeps publishing its last complete snapshot" while a window move is in flight. It may instead be intended under Pending demand does not hide the parent row. I couldn't find a rule for which of these applies to include demand that a window move creates.

To Reproduce

Standalone script, no framework: eager parents, so the window move needs no async load, and on-demand children whose loadSubset answers after 200 ms. Run with bun repro.ts.

repro.ts 
// bun repro.ts
import {
  BasicIndex,
  createCollection,
  createLiveQueryCollection,
  eq,
  extractSimpleComparisons,
  localOnlyCollectionOptions,
  materialize,
} from '@tanstack/db'

type Parent = { id: number }
type Child = { id: number; parentId: number; name: string }

const CHILD_LOAD_MS = 200
const indexed = { autoIndex: 'eager', defaultIndexType: BasicIndex } as const
const start = performance.now()
const log = (message: string) => console.log(`${(performance.now() - start).toFixed(0).padStart(4)} ms  ${message}`)

// Eager parents, so the window move itself needs no async load
const parents = createCollection(
  localOnlyCollectionOptions({
    ...indexed,
    getKey: (p: Parent) => p.id,
    initialData: Array.from({ length: 9 }, (_, i) => ({ id: i + 1 })),
  }),
)

// On-demand children, served by a backend that answers after CHILD_LOAD_MS
const backend: Array<Child> = Array.from({ length: 9 }, (_, i) => ({
  id: 101 + i,
  parentId: i + 1,
  name: `child-of-${i + 1}`,
}))

const children = createCollection<Child, number>({
  ...indexed,
  getKey: (c) => c.id,
  syncMode: 'on-demand',
  sync: {
    sync: ({ begin, write, commit, markReady }) => {
      markReady()
      return {
        loadSubset: async (options) => {
          const parentIds = new Set<number>()
          for (const f of extractSimpleComparisons(options.where)) {
            if (f.field.join('.') !== 'parentId') continue
            for (const v of f.operator === 'in' ? (f.value as Array<number>) : [f.value as number]) parentIds.add(v)
          }
          log(`  children.loadSubset parentIds=[${[...parentIds]}]`)
          await new Promise((r) => setTimeout(r, CHILD_LOAD_MS))
          begin()
          for (const c of backend.filter((c) => parentIds.has(c.parentId))) write({ type: 'insert', value: c })
          commit()
          log('  children.loadSubset resolved')
        },
      }
    },
  },
})

const page = createLiveQueryCollection({
  query: (q) =>
    q
      .from({ p: parents })
      .orderBy(({ p }) => p.id)
      .limit(3)
      .select(({ p }) => ({
        id: p.id,
        child: materialize(
          q
            .from({ c: children })
            .where(({ c }) => eq(c.parentId, p.id))
            .findOne(),
        ),
      })),
})

const show = () => `[${page.toArray.map((r) => `${r.id}:${r.child?.name ?? 'MISSING'}`).join(', ')}]`

page.subscribeChanges(() => log(`publish ${show()}  isLoadingSubset=${page.isLoadingSubset}`))

await page.preload()
log(`preload settled ${show()}`)

log('setWindow({ offset: 3, limit: 3 })')
const settlement = page.utils.setWindow({ offset: 3, limit: 3 })
if (settlement !== true) await settlement
log(`setWindow settled ${show()}`)

Output:

  26 ms    children.loadSubset parentIds=[1,2,3]
 230 ms  publish [1:child-of-1, 2:child-of-2, 3:child-of-3]  isLoadingSubset=false
 231 ms    children.loadSubset resolved
 231 ms  publish [1:child-of-1, 2:child-of-2, 3:child-of-3]  isLoadingSubset=false
 231 ms  preload settled [1:child-of-1, 2:child-of-2, 3:child-of-3]
 231 ms  setWindow({ offset: 3, limit: 3 })
 234 ms    children.loadSubset parentIds=[4,5,6]
 235 ms  publish [4:MISSING, 5:MISSING, 6:MISSING]  isLoadingSubset=true    <-- page published before its includes
 436 ms  publish [4:child-of-4, 5:child-of-5, 6:child-of-6]  isLoadingSubset=false
 436 ms    children.loadSubset resolved
 438 ms  setWindow settled [4:child-of-4, 5:child-of-5, 6:child-of-6]

Replacing materialize(...findOne()) with toArray(...) gives the same sequence, with [] instead of undefined.

Expected behavior

The same as the initial load: while the move is pending, keep publishing the last complete window ([1, 2, 3] with their children). Then publish [4, 5, 6] once, with their children, when the setWindow() promise settles.

If publishing the parents early is intended, it would help to document that for window moves, and for the hooks to expose that a move is pending. Today useLiveQuery's isLoading / isReady only cover the initial load, and isLoadingSubset isn't returned. An app can only hide the partial page by awaiting the setWindow() promise itself.

Real-world impact

A paginated table with numbered pages, over Electric on-demand collections. The query has an orderBy and materialize includes for the related rows, and the page is moved with setWindow (the workaround described in #2291). On every page change, the cells built from includes (reference, client, delivery date, a status computed from a related row) show their empty fallback for about 900 ms. Then they switch to the real values. A status derived from the related row can briefly be wrong, not just empty.

Environment

Additional context

  • #2291 / #2316: page moves through useLiveQuery. #2316 lists "new include-publication semantics" as out of scope, so this behavior would carry over to adapter-owned page windows.
  • #2131: the policy decision to hold the last complete snapshot while a window move is in flight.
  • #2156: includes are routed only from the rows the window shows. That is why a move creates new child demand at all.
Lenguaje dominante
TypeScript
Estrellas
4k
Forks
277
Merge medio
1 d 7 h
PR fusionados (30 d)
172

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de TanStack/db

Todos los issues de TanStack/db

Issues similares

Más issues de TypeScript

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.