Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Relation filters (semi/anti-joins) force a full-source load on orderBy+limit queries — forward them to loadSubset instead

Cerrado
  • ¿Algo no está bien?
#1,898 2 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
35/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
typescript
Área
backend-api-design, databases, performance

Línea de trabajo

Start in query/compiler/order-by.ts at processOrderBy, then trace OrderedSourceLoader, LoadSubsetOptions, loadSubset forwarding, and getLoadSubsetDemandKey. Run the prototype tests against fake on-demand backends, including honored and ignored relation hints and live child insert/delete cases. Done means limited relation-filtered queries avoid full-source loads while preserving exact pages and live-window updates.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Problem

A windowed live query (orderBy + limit) that filters its root rows by the existence of related rows loads the entire root collection.

Example: "seeks, newest first, 20 per page, that have at least one run by a user outside a hidden set". Expressed as a leftJoin of the child collection (or a distinct child subquery) plus where(not(isUndefined(j.key))) (semi-join), or isUndefined(j.key) (anti-join).

In query/compiler/order-by.ts (processOrderBy), requiresFullSource is set whenever any where references an alias other than the orderBy alias. OrderedSourceLoader then calls loadFullSource(), so the root's loadSubset receives an unlimited request with no limit/cursor. On-demand collections pull the whole table. Adapters that refuse unbounded loads return nothing.

Why it is there (as far as I can tell)

With a limit, the joined side loads lazily by key (SubsetDemandController). Root rows filtered out while their children are still in flight make dataNeeded positive, so loadMore keeps paging. With a slow child side this walks the whole table anyway, so forcing a full load is the safe fallback.

Proposal
  1. Structured, backend-agnostic hint. Add LoadSubsetOptions.relations?: Array<{ quantifier: 'some' | 'none'; collectionId: string; on: { parent: string[]; child: string[] }; where?: BasicExpression<boolean> }>. every(p) is none(not p). A collection may apply it (return only root rows that pass) or ignore it (return a superset), but must never drop a passing row. Local evaluation stays authoritative, so correctness never depends on the collection.
  2. Compiler. In processOrderBy, split the and conjuncts. Treat not(isUndefined(j.k)) / isUndefined(j.k) as a semi/anti-join when j is a LEFT join on eq(root.x, j.k) and j is a collection or a simple single-collection subquery. Those conjuncts no longer set requiresFullSource; they populate info.relations, which the root subscription forwards on every loadSubset (including requestLimitedSnapshot / tie loads).
  3. Loader. loadMore waits while any lazy join demand is unsettled or a joined subscription is loadingSubset. settleDemand already reruns the graph and the loaders.
  4. Joined-side changes. Changes on the joined side must also run the ordered loaders. Removing a child can shrink the window, and today that path (scheduleGraphRun(undefined)) passes no loader callback, so the window stays short.
  5. Dedupe. getLoadSubsetDemandKey includes relations, so a relation-narrowed load never dedupes (covers) the same load without the relation.

Backends can honor the hint exactly where they support it. For example, PostgREST: select=*,children!inner(userId)&children.userId=not.in.(…) for some, and a filtered embed with children=is.null for none. An SQL backend can use EXISTS (…).

Prototype

I implemented this as a patch on 0.9.2: about 150 lines of logic per build format, esm and cjs, plus types. It is covered by tests against fake on-demand backends, 40 root rows:

Case Today Hint honored Hint ignored
some, take 5 1 unlimited request, all 40 rows 2 requests, 6 rows 6 requests, 13 rows fetched, exact page
some, take 10 all 40 rows 2 requests, 11 rows 10 requests, 26 rows fetched, exact page
none / every, take 5 all 40 rows 2 requests, 6 rows 10 requests, 16 rows fetched, exact page

(The honored case's second request is the boundary-tie load.)

Tests cover:

  • collections that honor the hint (exact pages, no full-source load);
  • collections that ignore it (exact pages, stops at the minimal prefix, never loads the full source);
  • eager local-only collections;
  • live child insert and delete moving a parent into or out of the window, for some / none / every.

Without the demand gate in step 3, a slow child side (40 ms vs 0 ms) makes the ignoring case load 40 of 40 rows. Happy to open a PR from it if the direction looks right.

Out of scope / open questions
  • Should relations also flow for unlimited queries? The prototype only forwards them on limited windows.
  • Should demand-gating be scoped per join instead of query-wide? Query-wide is simple, but an unrelated include's pending demand delays paging (latency only).
  • Relation predicates under or / not, and joins through subqueries that themselves join, still force a full source in the prototype.
  • Adapters that page by offset count local rows, which other queries may have inflated. Relation-narrowed loads probably need to page by cursor.

Related: #1657 (loadSubset / pagination core RFC), #1887 (join key segments for remote sources).

Lenguaje dominante
TypeScript
Estrellas
3.9k
Forks
267
Merge medio
1 d 7 h
PR fusionados (30 d)
104

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de TanStack/db

Todos los issues de TanStack/db

Issues similares

Más issues de TypeScript

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.