Flaky timing-sensitive sync-service tests: ShapeCacheTest await/startup timeouts under CI load

Open
#4,712 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
52/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Quiet
Tech stack
elixir, github-actions, postgresql

Research direction

Start with the affected Electric.ShapeCacheTest, Electric.Plug.TraceContextPlugTest, and ShapeLogCollectorTest cases, focusing on their 5-second awaits and cleanup timing. Run the affected sync-service tests under the Postgres matrix and inspect the surrounding connection errors. Done means the tests remain meaningful, no longer flake under CI load, or are clearly tagged if the underlying saturation is not addressed.

Written by the indexing model from the issue text.

Description

Summary

A small family of timing-sensitive tests fails intermittently in the sync-service CI workflow — hopping between Postgres matrix versions, passing on rerun with no code change, and burying real signal for PR authors. Two recent PRs (#4711 and its review) each spent effort separating these from genuine failures.

Evidence

Electric.ShapeCacheTest — "await_snapshot_start/4 should wait for consumer to come up"

Electric.ShapeCacheTest — "get_or_create_shape_handle/2 against real db crashes when initial snapshot query fails to return data quickly enough"

Electric.Plug.TraceContextPlugTest — "electric member is found among other tracestate members"

ShapeLogCollectorTest — FlushTracker timing out in Repatch.cleanup

  • Reported during review of #4711 (first CI run of that branch)

Common signature

  • ~5s Task.await timeouts on tests that race consumer/snapshot startup
  • Postgres connection churn in the surrounding logs (tcp connect timeout, econnrefused, tcp recv (idle): closed) suggesting the shared PG instance is briefly saturated by concurrent async tests
  • Not reproducible locally on fast machines; version-hopping across the PG matrix

Suggested directions

  • Widen or parametrize the tight awaits in the affected tests (they're asserting eventual startup, not 5-second startup)
  • Or address the underlying PG saturation (connection pool sizing / readiness in the test env)
  • At minimum, tag the known-flaky set so authors and reviewers can distinguish them from real regressions without re-deriving this analysis

Filed while triaging CI on #4711, where these flakes twice muddied the signal.

Dominant language
TypeScript
Stars
10.4k
Forks
399
PR merge metrics
No merged PRs in 30d

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from electric-sql/electric

All issues in electric-sql/electric

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.