ShapeCache: coalesce concurrent create_or_wait callers via GenServer-owned per-shape lock + polling
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 48/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- elixir
- Domain
- backend, distributed-systems
Research direction
Start with Electric.ShapeCache.get_or_create_shape_handle/3, its create_or_wait_shape_handle handler, and ShapeCache init/1; review the PollWait API from #4371 first. Trace the existing safe_maybe_create_shape flow and ETS tables before adding the per-shape lock and polling path. Done means same-shape callers avoid mailbox buildup, crash recovery clears locks, and concurrent, backoff, and restart tests pass.
Written by the indexing model from the issue text.
Description
Background
Electric.ShapeCache.get_or_create_shape_handle/3 is called from Api.load_shape_info on every shape request that misses the in-process fast path. The fast path checks ShapeStatus.fetch_handle_by_shape (WriteBuffer ETS, then SQLite read connection) followed by fetch_latest_offset (Storage). On miss, the caller falls through to GenServer.call(ShapeCache, {:create_or_wait_shape_handle, …}, 30_000).
When N concurrent requests for the same uncached shape arrive, all of them miss the fast path and all of them enqueue a GenServer.call. Handler 1 does the actual creation work (safe_maybe_create_shape → add_shape + start_shape); handlers 2..N short-circuit via fetch_handle_by_shape_critical finding the shape that handler 1 just created. So the work is deduplicated, but every caller still occupies a mailbox slot for the full duration of handler 1's creation plus its own dispatch latency. Under filesystem tail latency, shape creation reaches multiple seconds and the queue depth is real.
This is a sibling of #4370 (EtsInspector) and #4371 (StatusMonitor) under the thundering-herd umbrella #4266. The cheap admission control (#4292 / #4291 / PR #4359) bounds the concurrent population that can reach ShapeCache at all via the :initial bucket, but does not prevent admitted requests for the same shape from piling into the ShapeCache mailbox.
Goal
Prevent the ShapeCache mailbox from accumulating concurrent calls for the same fresh shape. Only one caller per unique shape-creation event should issue a GenServer.call; every other caller should observe the in-progress state via ETS and poll until the result is ready.
Design
The lock is GenServer-owned for writes, caller-readable for routing decisions. Callers never write to the lock table, eliminating any stale-claim concerns.
Caller flow
1. fast_path(shape) hit? → return
2. :ets.member(shape_create_lock(stack_id), shape_hash)?
true → PollWait.until(poll_predicate, @call_timeout, <ShapeCache opts>)
false → GenServer.call(name(stack_id), {:create_or_wait_shape_handle, …}, @call_timeout)
Server handler (replaces today's body)
case fetch_handle_by_shape_critical(stack_id, shape) do
{:ok, handle} -> reply {handle, fetch_latest_offset(...)}
:error ->
:ets.insert(shape_create_lock(stack_id), {shape_hash, self()})
result = safe_maybe_create_shape(shape, state)
:ets.delete(shape_create_lock(stack_id), shape_hash)
reply result
end
Poll predicate
with {:ok, handle} <- ShapeStatus.fetch_handle_by_shape(stack_id, shape),
true <- ShapeStatus.shape_has_been_activated?(stack_id, handle),
{:ok, offset} <- fetch_latest_offset(stack_id, handle) do
{:ready, {handle, offset}}
else
_ -> :not_ready
end
shape_has_been_activated?/2 checks the last_used_timestamp that update_last_read_time_to_now/2 sets at the very end of start_shape, so it serves as a "creation fully succeeded" signal. If creation fails, clean_shape removes the row and waiters never observe :ready, eventually timing out.
Properties
- Mailbox growth bounded by a single race window. Once a shape's lock is set, all new callers bypass the GenServer entirely. The mailbox grows only during the window between "first caller's call queued" and "handler dequeues and sets the lock" — i.e. mailbox dispatch latency in the common case, the previous shape's creation time in the contended case (and even then, capped at "one entry per fast-path-miss within the window").
- No
:waitreply needed. Any caller whose call reaches the handler with the lock unset is either the creator (does the work) or finds the shape already exists via the critical fetch (reply with the cached result). There's no third response code. - No stale-lock recovery needed. Callers don't write the lock, so they can't strand it. The GenServer always clears the lock in the same handler invocation that set it. On GenServer restart,
init/1wipes the lock table. - Specific error reasons are lost for polling waiters. They get
:timeoutafter the deadline. Same trade-off as #4370 / #4371.
Backoff considerations — PollWait must accept per-caller config
This is the part to be aware of when implementing.
The defaults in #4371 (initial 25ms → doubling → 500ms cap) are tuned for stack-readiness waiting, where the underlying event flips on a second-to-minute timescale and aggressive polling is wasted. ShapeCache is the opposite: a single shape creation completes in tens to low-hundreds of milliseconds in the common case. Polling at a 500ms cap would routinely add hundreds of milliseconds of latency for no benefit.
ShapeCache's caller should pass its own backoff:
PollWait.until(poll_predicate, @call_timeout,
initial_interval: 5,
max_interval: 100,
backoff: 2.0
)
Schedule: 5 → 10 → 20 → 40 → 80 → 100 → 100 … ms. Most shape creations are answered within the first 2–3 polls.
The PollWait primitive proposed in #4371 already accepts these as opts, so no primitive change is needed — only that the implementation honour the per-call overrides without baking the StatusMonitor defaults into a global config.
Crash safety
If the GenServer crashes mid-creation, its state is lost but the named, public lock table persists. On restart, init/1 runs :ets.delete_all_objects(shape_create_lock(stack_id)) to wipe stale entries. Any in-flight waiters that were polling time out after their existing deadline; new requests start fresh.
Suggested execution
- Depends on
Electric.PollWaitlanding as part of #4371. - Add the per-shape lock ETS table (
shape_create_lock:<stack_id>), created in ShapeCache'sinit/1with:public, :set, :named_table, read_concurrency: true, write_concurrency: :auto. Cleared ininit/1to handle crash recovery. - Modify
get_or_create_shape_handle/3to add the:ets.membercheck beforeGenServer.call, with the polling fallback. - Modify the
handle_call({:create_or_wait_shape_handle, …}, …)body to set/clear the lock aroundsafe_maybe_create_shape/2. - Test:
- Concurrent fan-out for the same fresh shape: assert mailbox depth is bounded by the brief-window count, and N–1 waiters receive the result via polling.
- Crash recovery: kill the GenServer mid-creation, assert lock table is empty after restart, assert new requests succeed.
- Backoff: assert poll cadence matches the configured schedule (within jitter).
- Dominant language
- TypeScript
- Stars
- 10.4k
- Forks
- 399
- PR merge metrics
- No merged PRs in 30d
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from electric-sql/electric
-
camelToSnake and snakeCamelMapper() drop the underscore when a digit precedes a camelCase boundaryOpen
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
electric-sql/electric#4855 ·
-
ShapeStream does not notify subscribers when it gives up on a non-Error valuePossibly taken @Tyagiquamar claimed this 19 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
electric-sql/electric#4830 ·
-
@electric-sql/client throws SecurityError on import when the browser blocks site storage (typeof localStorage outside try)Possibly taken @mohanedmoh claimed this 37 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
electric-sql/electric#4826 · 1 comment ·
-
agents-chat-starter: spawn PUT goes to the durable-streams proxy, so no entity is created and agents never wakePossibly taken @adityavkk claimed this 133 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
electric-sql/electric#4687 ·
-
Listen IP addressOpenfeature request triage
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
electric-sql/electric#4150 · 1 comment ·
All issues in electric-sql/electric
Similar issues
-
area:cli area:security bug priority:medium
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
"code source" and "numerique.gouv" links: "new window" mention duplicated in the accessible nameOpenaccessibility
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
numerique-gouv/lasuite-landingpage#315 ·
Maintainers usually reply within 1 day
-
needs:triage
Difficulty 1/5 Under an hour Newbie friendliness 92/100
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 72/100
HyeokjaeChoi/vaultme#585 ·
Maintainers usually reply within 1 day
-
enhancement priority/P2
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
beettlle/pi-smart-router#200 ·