Shape id reuse after engine restart can append onto stale shape/<id> streams
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 52/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- rust
- Domain
- backend, distributed-systems
Research direction
Start with the counter and create_shape paths in apps/engine/src/engine.rs:291, 473-475, then inspect ensure_stream in apps/engine/src/ds.rs:75-92. Reproduce a restart with surviving shape/* streams and verify that a new shape cannot append to stale rows; boot recovery should also address orphaned streams if that remains in scope.
Written by the indexing model from the issue text.
Description
Found during the production-readiness investigation (vs upstream sync-service).
Symptom / failure scenario: shape ids come from an in-memory counter that resets to 1 on restart (apps/engine/src/engine.rs:291, 473-475), while shape/* streams from the previous run survive on the durable-streams server. ensure_stream is an idempotent PUT (apps/engine/src/ds.rs:75-92), so the first post-restart create_shape reuses shape/s1 and appends a fresh backfill on top of the old run's rows. A client reading that stream from offset=-1 sees stale rows merged with the new shape's contents — silent data corruption, no error anywhere.
Fix direction: make shape ids collision-free across restarts (persist the counter, or random/uuid ids), and garbage-collect orphaned shape/* streams at boot. Both fall out of the shape-catalog/recovery epic, but the id collision is a standalone corruption hazard worth closing first.
🤖 Generated with Claude Code
- Dominant language
- Rust
- Stars
- 31
- Forks
- 6
- PR merge metrics
- No merged PRs in 30d
Getting set up
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from electric-sql/electric-circuits
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
All issues in electric-sql/electric-circuits
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
trailofbits/dylint#2107 ·
Maintainers usually reply within 1 day
-
area:cli bug good first issue priority:medium
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
Maintainers usually reply within 1 day
-
arrays_zip with two same-named inputs fails with "ArrowArray struct has 2 children (expected 1)"Openbug requires-triage
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
apache/datafusion-comet#6251 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
bug false-positive harper-core linting
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Automattic/harper#4471 ·
Maintainers usually reply within 1 day