Evict stale partition mappings for vanished/detached relations in Partitions.handle_relation/2
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 68/100
Research direction
Start in packages/sync-service/lib/electric/shapes/partitions.ex, reading handle_relation/2, expand_change/2, and the clean_up_partitions/1 semantics. Inspect the existing partition and ShapeLogCollector tests before adding a regression covering drop or detach, same-name recreation as a plain table, and change expansion. Done means stale mappings are removed without desynchronizing partition ownership or active counts, and the recreated table's change is not sent to the old root shape.
Written by the indexing model from the issue text.
Description
Summary
Electric.Shapes.Partitions.handle_relation/2 does not evict stale entries from the partitions map when a relation stops being a tracked partition. Because handle_txn_fragment/2 expands changes by {schema, table} name, a stale mapping can silently mis-route changes to the wrong root shape after a drop/detach + same-name recreation.
Surfaced by a Codex review on #4565 (commit a4dcc7d61), but the behavior is pre-existing — that PR only made the :table_not_found path reachable without crashing.
The bug
The partition map maps partition_table → [root_table], and expansion looks it up by name:
defp expand_change(%{relation: relation} = change, state) do
[change | state.partitions |> Map.get(relation, []) |> Enum.map(&%{change | relation: &1})]
end
Two branches of handle_relation/2 return the state unchanged without clearing a previously-tracked entry for that name:
# packages/sync-service/lib/electric/shapes/partitions.ex
{:ok, _} ->
{:ok, state} # relation no longer has a parent (detached / never partitioned)
:table_not_found ->
# the table was dropped (or dropped and recreated under a new oid) ...
{:ok, state} # added in #4565
If a tracked partition P of root R is dropped (or detached) and a new table with the same {schema, table} name is later created as a non-partition, the stale partitions[P] = [R] entry persists. Changes to the new P then get duplicated onto the old root shape R, which is incorrect.
Why it's narrow
It requires all of:
- an active shape on the partitioned root
R, - partition
Pdropped/detached, - a new relation recreated with the exact same schema-qualified name,
- recreated as a plain/non-partition table (if recreated as a partition, the
{:ok, %{parent: ...}}branch overwrites the stale entry, so no bug), - changes flowing to the recreated table.
Fix considerations
-
Evict the vanished/detached relation from
partitionsin both the:table_not_foundand the no-parent{:ok, _}branches — fixing only one (as the inline suggestion did) leaves the identical hole in the other. -
Eviction must be ownership-aware. Entries in
partitionscome from two places with different bookkeeping:add_shape/3— also tracked inpartition_ownershipand counted inactive,handle_relation/2discovery — not tracked inpartition_ownership.
A blind
Map.delete/2on a relation message could desyncpartitionsfrompartition_ownership/activefor shape-owned children, so the cleanup needs to account for ownership (likely reuse / extendclean_up_partitions/1semantics rather than delete directly). -
Add a regression test that reproduces drop/detach → recreate-as-plain-table → asserts the change is not duplicated to the old root shape.
References
- PR #4565 — "Don't crash ShapeLogCollector on introspection failures in the replication path"
- File:
packages/sync-service/lib/electric/shapes/partitions.ex(handle_relation/2,expand_change/2)
- Dominant language
- TypeScript
- Stars
- 10.4k
- Forks
- 399
- PR merge metrics
- No merged PRs in 30d
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from electric-sql/electric
-
camelToSnake and snakeCamelMapper() drop the underscore when a digit precedes a camelCase boundaryOpen
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
electric-sql/electric#4855 ·
-
ShapeStream does not notify subscribers when it gives up on a non-Error valuePossibly taken @Tyagiquamar claimed this 19 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
electric-sql/electric#4830 ·
-
@electric-sql/client throws SecurityError on import when the browser blocks site storage (typeof localStorage outside try)Possibly taken @mohanedmoh claimed this 37 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
electric-sql/electric#4826 · 1 comment ·
-
agents-chat-starter: spawn PUT goes to the durable-streams proxy, so no entity is created and agents never wakePossibly taken @adityavkk claimed this 133 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
electric-sql/electric#4687 ·
-
Listen IP addressOpenfeature request triage
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
electric-sql/electric#4150 · 1 comment ·
All issues in electric-sql/electric
Similar issues
-
area:cli area:security bug priority:medium
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
"code source" and "numerique.gouv" links: "new window" mention duplicated in the accessible nameOpenaccessibility
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
numerique-gouv/lasuite-landingpage#315 ·
Maintainers usually reply within 1 day
-
needs:triage
Difficulty 1/5 Under an hour Newbie friendliness 92/100
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 72/100
HyeokjaeChoi/vaultme#585 ·
Maintainers usually reply within 1 day
-
enhancement priority/P2
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
beettlle/pi-smart-router#200 ·