Hacktoberfest 2026: as issues que os mantenedores marcaram para outubro, abertas e boas para iniciantes. Ver issues do Hacktoberfest

Electric progressive: apply and confirm changes to visible rows during the background full sync

Aberta
  • Algo errado?
#2,120 0 comentários 0 reações 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 1 dia

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
5/5
Tempo estimado
Mais de uma semana
Facilidade para iniciantes
30/100
Tipo de issue
Funcionalidade
Clareza
Razoavelmente clara
Status de atividade
Ativa
Stack de tecnologia
typescript
Domínio
backend, database

Direção de pesquisa

Start with packages/electric-db-collection/src/electric.ts and the progressive witnesses in electric-live-query.test.ts; trace how buffered changes, snapshot metadata, txid evidence, and the atomic swap interact. Use the recovery and descriptor-isolation oracles to check ordering around snapshot xmin, xmax, and xip, then inspect persistence and reset behavior. Done means visible-row changes and their txids are applied before the swap, with no stale rollback or duplicate application, and the stated evidence passes.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

Summary

During a progressive collection's background full sync, Electric buffers stream changes and their txids until the atomic swap. #2115 makes a live query over a progressive source ready as soon as its own subset loads. A user can now edit a visible row while the full sync is still running. That mutation's awaitTxId cannot resolve until the swap. If the full sync takes longer than the awaitTxId timeout, the handler throws and the optimistic change rolls back.

The stream starts at once, so Electric receives the confirming change long before the swap. Electric could apply that change and its txid during buffering for rows that the client already shows.

Current behavior

In packages/electric-db-collection/src/electric.ts:

  • While isBufferingInitialSync() is true, change messages go to bufferedMessages. Their txids are not added to newTxids. The comment says "txids will be extracted during atomic swap".
  • The atomic swap on the first up-to-date applies the buffered changes, extracts their txids, and then resolves txid waiters through publishEvidence.

As a result:

  1. Confirmation waits for the full sync. awaitTxId(txid) for a write to a visible row resolves only at the swap.
  2. Visible rows go stale. A ready progressive query shows its subset snapshot rows unchanged until the swap. #2115 records this as a known limit, and electric-live-query.test.ts pins it ("a ready query's rows update only when the full sync completes").

Before #2115, useLiveQuery already rendered rows in this window. Suspense users now reach it too.

Proposal

Apply stream changes during buffering for keys that a subset snapshot already loaded, when the change is newer than that snapshot. Record their txids when the change applies. Keep buffering every other change until the swap.

Electric has the needed pieces:

  • progressiveSnapshotRows keeps the snapshot row for a key when a later stream update needs that row as its baseline.
  • Each fetchSnapshot returns snapshot metadata, and isVisibleInSnapshot can order a txid against it.

The txid must resolve only when its change is visible. If Electric confirms a change that it still buffers, the optimistic state drops and the old snapshot row shows again until the swap.

Questions to settle

  • Ordering. A change can be older than, or included in, the snapshot that loaded its key. Use the snapshot's xmin, xmax, and xip list to decide whether to apply the change, skip it as already included, or keep buffering it.
  • The swap. The swap must not apply an early-applied change a second time, and it must not move a row back to an older value.
  • Keys outside any subset. Changes to keys that no subset loaded stay buffered, so their txids stay unconfirmed until the swap. Decide whether awaitTxId should report this case, or whether a write to a row the client never loaded is outside scope.
  • Must-refetch. A reset during buffering discards the subset rows. Changes applied early before the reset must follow the reset's truncate rules.
  • Persistence. Early-applied changes go through the same commit and receipt path. Check resume metadata and seenTxids evidence for persisted collections.

Evidence to add

  • Extend the progressive witnesses in electric-live-query.test.ts. A write to a visible row during buffering must become visible, and its awaitTxId must resolve before the swap.
  • Flip the pinned freshness test on purpose, and update the limit in packages/db/src/query/live/ARCHITECTURE.md and in the #2115 review record.
  • Add an ordering lane with changes before, inside, and after the loading snapshot, checked against an independent model. The Electric recovery and descriptor-isolation oracles already generate progressive histories and are a good place to start.
  • A hostile control: a design that confirms a txid without applying its change must fail at the first checkpoint after confirmation.

Related: #2113, #2115, #2118.

Linguagem predominante
TypeScript
Estrelas
3.9k
Forks
272
Merge médio
1d 48min
PRs com merge (30d)
238

Preparar o ambiente

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de TanStack/db

Todas as issues de TanStack/db

Issues semelhantes

Mais issues de TypeScript

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.