Persistence: a follower shows its leader's rows after a cache-claim rotation

Start on the branch fix/persisted-follower-rotation (03b8c15ee) and read persisted.ts around scoped recovery and the commit-notice handling. The oracle lanes are in packages/db-sqlite-persistence-core/tests/persisted-oracle.test.ts, which you can run to see which mutant cells fail. Done means the open questions are resolved, the six mutants are still killed, and persisted.ts is reduced in code weight.

Summary

A follower tab whose persisted cache claim rotates can fail to show its leader's rows. #2128 fixes the follower's load ordering. Its review rounds also found a group of follower bugs that occur only around a cache-claim rotation. That work is too large to ship with #2128, so it moved to this issue. The work so far is on the branch fix/persisted-follower-rotation (03b8c15ee).

The problem

When a follower's cache claim rotates, the follower enters scoped recovery. Scoped recovery assumes that the Collection's own source supplies fresh rows, so it:

• refuses durable rows (loadSubsetRowsUnsafe returns [] during scoped recovery).
• turns every commit notice into another refetch (invalidateFromCommittedTxUnsafe calls refreshScopedSubsets).

A follower has no source of its own. Its rows arrive from the leader as durable commits and notices, and scoped recovery refuses both. The follower therefore never shows the leader's rows. Against a leader that commits on every ensure, each refetch causes another commit and another notice, so the follower loops. On main, the probe showed 10 ensures in 50 ms and no rows.

The design on the branch

The maintainer chose a root-cause design over threading an "owner evidence" flag through every reload path:

1. While a rotated follower's reacquisition ensures are in flight, it holds the new generation's commit notices.
2. When every active demand reacquires, it ends scoped recovery for that generation and applies the held notices through the normal path.
3. If a reacquisition fails, scoped recovery stays, and the source reports the failure.
4. Each scoped recovery owns its own reacquisition record. A newer recovery finishes the older record.
5. Recovery can end only if this recovery rotated into a fresh generation and the node follows a leader.
6. A demand that is released or aborted during reacquisition is ignored, and its AbortError is not reported.
7. A load that starts during reacquisition waits until reacquisition ends and its commits apply.

This design needs no getLeaderId() on the coordinator, so it has no public API change.

Bugs that the review rounds found and that the branch fixes

• The reconciliation reload, the seq-gap recovery, and the reset-term fence dropped the notice's evidence, so the follower looped or took an extra refetch round.
• A recovery with no rotation and no active demand ended scoped recovery at once and trusted the old cache again.
• A second rotation during reacquisition, with a leadership change, could leave notices held forever.
• A demand released during reacquisition kept scoped recovery in place and reported a false sync error.
• A load during reacquisition could settle while the leader's commits were still held.
• A queue flush could process held notices during reacquisition. In one topology the follower spun without yielding, on main as well.

Open questions

• A new demand after a superseded rotation as leader. In the cell "superseded as leader", a new demand c loads and settles. It ends with no row. The node stays in scoped recovery after it rotates as leader, and the owner wrote c into the superseded generation. The branch changed the cell to check only that the load settles. This is unproven. Before anyone accepts that expectation, decide whether it follows from the law or is a bug.
• No retry after a failed reacquisition. Scoped recovery stays until a later rotation or cleanup. The lane prose records this as a limit. Decide whether this is the intended behavior.
• Code weight. persisted.ts on the branch is +119 net code lines against main. Look for reductions before review.

Oracle coverage on the branch

packages/db-sqlite-persistence-core/tests/persisted-oracle.test.ts has these lanes:

• persistence.follower-generation-notice: commit before or after rotation, then a leader update, a gap notice, a reconciliation notice, or a peer notice.
• Rotated reacquisition: two demands reacquire in either order, and an owner update can arrive between them. Any reacquisition can fail or be released. A new demand can start during reacquisition, and a rotation can be superseded as leader.

The branch kills these mutants:

Mutant	Cells that fail
Notices are not held	19
Scoped recovery never ends	21
Recovery ends before every reacquisition settles	19
Recovery ends after a failure	25
A load does not wait during reacquisition	20
A superseded record never finishes	18

#2128 keeps these lanes and pins the cells that its cut-down scope does not satisfy, with a comment that points to this issue.

Next steps

1. Rebase or merge fix/persisted-follower-rotation onto main after #2128 merges.
2. Resolve the open questions above.
3. Run a code review and an oracle review that reruns the mutants independently.
4. Reduce code weight, then open the pull request.

Related: #2128, #2115.