Hacktoberfest 2026 : les issues que les mainteneurs ont marquées pour octobre, ouvertes et accessibles aux débutants. Parcourir les issues Hacktoberfest

Relation filters (semi/anti-joins) force a full-source load on orderBy+limit queries — forward them to loadSubset instead

Fermée
  • Quelque chose ne va pas ?
#1,898 2 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 1 jour

Personne n'a encore pris cette issue.

Évaluation

Difficulté
5/5
Temps estimé
Plus d'une semaine
Accessibilité débutants
35/100
Type d'issue
Fonctionnalité
Clarté
Plutôt claire
Activité
Active
Stack technique
typescript
Domaine
backend-api-design, databases, performance

Piste de recherche

Start in query/compiler/order-by.ts at processOrderBy, then trace OrderedSourceLoader, LoadSubsetOptions, loadSubset forwarding, and getLoadSubsetDemandKey. Run the prototype tests against fake on-demand backends, including honored and ignored relation hints and live child insert/delete cases. Done means limited relation-filtered queries avoid full-source loads while preserving exact pages and live-window updates.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

Problem

A windowed live query (orderBy + limit) that filters its root rows by the existence of related rows loads the entire root collection.

Example: "seeks, newest first, 20 per page, that have at least one run by a user outside a hidden set". Expressed as a leftJoin of the child collection (or a distinct child subquery) plus where(not(isUndefined(j.key))) (semi-join), or isUndefined(j.key) (anti-join).

In query/compiler/order-by.ts (processOrderBy), requiresFullSource is set whenever any where references an alias other than the orderBy alias. OrderedSourceLoader then calls loadFullSource(), so the root's loadSubset receives an unlimited request with no limit/cursor. On-demand collections pull the whole table. Adapters that refuse unbounded loads return nothing.

Why it is there (as far as I can tell)

With a limit, the joined side loads lazily by key (SubsetDemandController). Root rows filtered out while their children are still in flight make dataNeeded positive, so loadMore keeps paging. With a slow child side this walks the whole table anyway, so forcing a full load is the safe fallback.

Proposal
  1. Structured, backend-agnostic hint. Add LoadSubsetOptions.relations?: Array<{ quantifier: 'some' | 'none'; collectionId: string; on: { parent: string[]; child: string[] }; where?: BasicExpression<boolean> }>. every(p) is none(not p). A collection may apply it (return only root rows that pass) or ignore it (return a superset), but must never drop a passing row. Local evaluation stays authoritative, so correctness never depends on the collection.
  2. Compiler. In processOrderBy, split the and conjuncts. Treat not(isUndefined(j.k)) / isUndefined(j.k) as a semi/anti-join when j is a LEFT join on eq(root.x, j.k) and j is a collection or a simple single-collection subquery. Those conjuncts no longer set requiresFullSource; they populate info.relations, which the root subscription forwards on every loadSubset (including requestLimitedSnapshot / tie loads).
  3. Loader. loadMore waits while any lazy join demand is unsettled or a joined subscription is loadingSubset. settleDemand already reruns the graph and the loaders.
  4. Joined-side changes. Changes on the joined side must also run the ordered loaders. Removing a child can shrink the window, and today that path (scheduleGraphRun(undefined)) passes no loader callback, so the window stays short.
  5. Dedupe. getLoadSubsetDemandKey includes relations, so a relation-narrowed load never dedupes (covers) the same load without the relation.

Backends can honor the hint exactly where they support it. For example, PostgREST: select=*,children!inner(userId)&children.userId=not.in.(…) for some, and a filtered embed with children=is.null for none. An SQL backend can use EXISTS (…).

Prototype

I implemented this as a patch on 0.9.2: about 150 lines of logic per build format, esm and cjs, plus types. It is covered by tests against fake on-demand backends, 40 root rows:

Case Today Hint honored Hint ignored
some, take 5 1 unlimited request, all 40 rows 2 requests, 6 rows 6 requests, 13 rows fetched, exact page
some, take 10 all 40 rows 2 requests, 11 rows 10 requests, 26 rows fetched, exact page
none / every, take 5 all 40 rows 2 requests, 6 rows 10 requests, 16 rows fetched, exact page

(The honored case's second request is the boundary-tie load.)

Tests cover:

  • collections that honor the hint (exact pages, no full-source load);
  • collections that ignore it (exact pages, stops at the minimal prefix, never loads the full source);
  • eager local-only collections;
  • live child insert and delete moving a parent into or out of the window, for some / none / every.

Without the demand gate in step 3, a slow child side (40 ms vs 0 ms) makes the ignoring case load 40 of 40 rows. Happy to open a PR from it if the direction looks right.

Out of scope / open questions
  • Should relations also flow for unlimited queries? The prototype only forwards them on limited windows.
  • Should demand-gating be scoped per join instead of query-wide? Query-wide is simple, but an unrelated include's pending demand delays paging (latency only).
  • Relation predicates under or / not, and joins through subqueries that themselves join, still force a full source in the prototype.
  • Adapters that page by offset count local rows, which other queries may have inflated. Relation-narrowed loads probably need to page by cursor.

Related: #1657 (loadSubset / pagination core RFC), #1887 (join key segments for remote sources).

Langage dominant
TypeScript
Étoiles
4k
Forks
277
Merge moyen
17 h 33 min
PR mergées (30 j)
227

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de TanStack/db

Toutes les issues de TanStack/db

Issues similaires

Plus d'issues TypeScript

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.