Hacktoberfest 2026: die Issues, die Maintainer für den Oktober markiert haben – offen und einsteigerfreundlich. Hacktoberfest-Issues durchsuchen

Electric progressive: apply and confirm changes to visible rows during the background full sync

Offen
  • Stimmt etwas nicht?
#2,120 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
5/5
Geschätzter Aufwand
Über eine Woche
Anfängerfreundlichkeit
30/100
Issue-Typ
Feature
Klarheit
Größtenteils klar
Aktivitätsstatus
Aktiv
Tech-Stack
typescript
Bereich
backend, database

Rechercherichtung

Start with packages/electric-db-collection/src/electric.ts and the progressive witnesses in electric-live-query.test.ts; trace how buffered changes, snapshot metadata, txid evidence, and the atomic swap interact. Use the recovery and descriptor-isolation oracles to check ordering around snapshot xmin, xmax, and xip, then inspect persistence and reset behavior. Done means visible-row changes and their txids are applied before the swap, with no stale rollback or duplicate application, and the stated evidence passes.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

Summary

During a progressive collection's background full sync, Electric buffers stream changes and their txids until the atomic swap. #2115 makes a live query over a progressive source ready as soon as its own subset loads. A user can now edit a visible row while the full sync is still running. That mutation's awaitTxId cannot resolve until the swap. If the full sync takes longer than the awaitTxId timeout, the handler throws and the optimistic change rolls back.

The stream starts at once, so Electric receives the confirming change long before the swap. Electric could apply that change and its txid during buffering for rows that the client already shows.

Current behavior

In packages/electric-db-collection/src/electric.ts:

  • While isBufferingInitialSync() is true, change messages go to bufferedMessages. Their txids are not added to newTxids. The comment says "txids will be extracted during atomic swap".
  • The atomic swap on the first up-to-date applies the buffered changes, extracts their txids, and then resolves txid waiters through publishEvidence.

As a result:

  1. Confirmation waits for the full sync. awaitTxId(txid) for a write to a visible row resolves only at the swap.
  2. Visible rows go stale. A ready progressive query shows its subset snapshot rows unchanged until the swap. #2115 records this as a known limit, and electric-live-query.test.ts pins it ("a ready query's rows update only when the full sync completes").

Before #2115, useLiveQuery already rendered rows in this window. Suspense users now reach it too.

Proposal

Apply stream changes during buffering for keys that a subset snapshot already loaded, when the change is newer than that snapshot. Record their txids when the change applies. Keep buffering every other change until the swap.

Electric has the needed pieces:

  • progressiveSnapshotRows keeps the snapshot row for a key when a later stream update needs that row as its baseline.
  • Each fetchSnapshot returns snapshot metadata, and isVisibleInSnapshot can order a txid against it.

The txid must resolve only when its change is visible. If Electric confirms a change that it still buffers, the optimistic state drops and the old snapshot row shows again until the swap.

Questions to settle

  • Ordering. A change can be older than, or included in, the snapshot that loaded its key. Use the snapshot's xmin, xmax, and xip list to decide whether to apply the change, skip it as already included, or keep buffering it.
  • The swap. The swap must not apply an early-applied change a second time, and it must not move a row back to an older value.
  • Keys outside any subset. Changes to keys that no subset loaded stay buffered, so their txids stay unconfirmed until the swap. Decide whether awaitTxId should report this case, or whether a write to a row the client never loaded is outside scope.
  • Must-refetch. A reset during buffering discards the subset rows. Changes applied early before the reset must follow the reset's truncate rules.
  • Persistence. Early-applied changes go through the same commit and receipt path. Check resume metadata and seenTxids evidence for persisted collections.

Evidence to add

  • Extend the progressive witnesses in electric-live-query.test.ts. A write to a visible row during buffering must become visible, and its awaitTxId must resolve before the swap.
  • Flip the pinned freshness test on purpose, and update the limit in packages/db/src/query/live/ARCHITECTURE.md and in the #2115 review record.
  • Add an ordering lane with changes before, inside, and after the loading snapshot, checked against an independent model. The Electric recovery and descriptor-isolation oracles already generate progressive histories and are a good place to start.
  • A hostile control: a design that confirms a txid without applying its change must fail at the first checkpoint after confirmation.

Related: #2113, #2115, #2118.

Vorherrschende Sprache
TypeScript
Sterne
3.9k
Forks
272
Ø Merge
1 T. 1 Std.
Gemergte PRs (30 T.)
235

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus TanStack/db

Alle Issues in TanStack/db

Ähnliche Issues

Weitere Issues zu TypeScript

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.