Entities become silently unreachable after `deleteEntity`+respawn or `agents stop`/`start` — stale wake/runner bindings
まだ誰も着手していません。
評価
- 難易度
- 5/5
- 見積もり時間
- 1週間以上
- 初心者へのやさしさ
- 45/100
- issue の種類
- バグ
- 明瞭さ
- おおむね明確
- 活発さ
- 静か
- 技術スタック
- docker, postgresql, typescript
調査の方向性
まず、electric-ax agents CLI と、registerTypes()、spawnEntity()、deleteEntity()、sendEntityMessage() への bridge 呼び出しを通じて、両方のトリガーを再現します。agents-server の wake/runner ライフサイクルとそのプロセスメモリ状態を、一覧にある PostgreSQL テーブルと併せて追跡します。再起動と同じ URL での respawn によって live binding が維持され、古い状態がクリーンアップされ、live binding のないメッセージがサイレントな 204 を受け取らないことが完了条件です。
索引モデルが issue の本文から書いたものです。
説明
Summary
There are two ways to leave a spawned entity silently unreachable. In both, the entity row exists in postgres and the agents UI shows the entity as present, but messages sent to it are never processed. The two have different triggers and slightly different failure signatures, but they look like the same underlying problem: the wake/runner binding for an entity URL is not reconciled, so the entity is "alive" in storage but has nothing wired up to actually run its handler.
Filing them together because a fix for one likely addresses the other.
Environment
@electric-ax/agents-runtime0.2.2@electric-ax/durable-streams-client-beta0.3.1agents-server0.4.x (localelectric-ax agentsDocker stack)electric-axCLI 0.2.2- macOS, Docker Desktop
Setup in all repros: a long-running host process ("the bridge") that calls registerTypes() to register entity types against its own webhook URL, then spawns entities and relays messages to them.
Trigger 1 — entities unreachable after agents stop && agents start (no volume wipe)
Repro
- Start the stack:
electric-ax agents start. - From the bridge:
registerTypes(), thenspawnEntity({ type, id }). Verify the entity wakes and processes asendEntityMessage(works). electric-ax agents stop(no--remove-volumes).electric-ax agents start.- Restart the bridge process.
sendEntityMessageto the entity → server returns HTTP 204, but no wake is ever dispatched to the bridge webhook. The entity never runs. A direct HTTP send to the entity's inbox endpoint behaves the same: 204, silent, no wake.
Scope
Only entities that existed at the moment of agents stop are affected. An entity spawned after the restart works normally — until the next stop.
Diagnostic that isolated it
Changing the entity id to a value that had never been used, then restarting the bridge, produced a fresh entity that woke normally. We then spawned a new entity inside that working post-restart session — it worked — and ran agents stop && agents start a second time. The new entity was now also dead. So the discriminator is the substrate stop event itself, not how recently the entity was spawned or which bridge process created it.
Reading
Entity state in postgres survives the restart intact. The wake-registration that binds "entity URL → webhook URL" is not rebound to the restarted bridge's webhook on agents start, so wake dispatch never fires.
Trigger 2 — entities unreachable after deleteEntity + respawn on the same URL
Repro
- Start the stack and spawn an entity; confirm it wakes on
sendEntityMessage(works). client.deleteEntity('/type/id')→ returns 2xx success.client.spawnEntity({ type, id })again on the same URL → returns success, no error.sendEntityMessageto that URL →409 {"code":"NOT_RUNNING","message":"Entity is stopped"}.
No agents stop is involved — this happens within a single substrate uptime.
What we investigated
- Postgres is not cleaned by
deleteEntity. After step 2,SELECT url FROM entities WHERE url LIKE '/type/%'still returns the row. - A direct SQL purge does not fix it. We
DELETEd the URL's rows directly across all 8 per-entity tables (entities,wake_registrations,entity_dispatch_state,entity_manifest_sources,scheduled_tasks,tag_stream_outbox,wake_notifications,consumer_claims), then respawned. The send still returns409. entity_dispatch_stateis half-initialized after respawn. The row exists, but everyactive_*column (active_consumer_id,active_runner_id,active_epoch,active_claimed_at,active_lease_expires_at) isNULL— no runner is bound.- Only a process restart clears it.
electric-ax agents stop --remove-volumes && electric-ax agents startlets a fresh spawn on the same URL work again. Since a clean postgres alone (via the SQL purge above) does not fix it, the broken state appears to live in agents-server process memory, not in postgres.
Common thread
In both cases: the entity row exists, postgres looks normal, the agents UI shows the entity — but no runner/wake binding is actually live for it. A full process restart of agents-server is the only thing that reliably clears the bad state.
Worst-shaped part
Neither failure surfaces at spawn time. spawnEntity returns success and the agents UI shows the entity green. Trigger 1 is the worse of the two: an inbox write to an orphaned entity returns HTTP 204 — a success response — and the message is silently dropped. Application code has no signal that the message will never be processed. Trigger 2 at least returns an explicit 409.
Suggested fixes
- On
agents start, walk the entity registry and re-validate wake-registrations against the webhook URLs registered by the currentregisterTypes()calls. Rebind rows whose target entity still exists; drop rows whose entity is gone. Log a summary of rehydrated vs. dropped registrations. - Make
deleteEntitytruly destructive — clear postgres rows, in-process cache, and wake/runner bindings — so a URL is genuinely free for reuse; or makespawnEntityon a stale URL re-initialize the in-process lifecycle state and bind a runner. - At minimum: an inbox write or
sendEntityMessageto an entity with no live wake binding should return a loud error (e.g.409/503), never a silent204. A successful HTTP response should mean the message will actually be dispatched.
Workaround
In application code: never reuse an entity URL. Generate a unique suffix per session (e.g. /type/id-<sessionId>); old entities become dead rows in postgres but never collide with new ones. Also avoid agents stop && agents start on a substrate with live entities you still need. Both workarounds leak dead entity rows over time.
- 主要言語
- TypeScript
- スター
- 10.4k
- フォーク
- 399
- PR マージ指標
- 30日以内にマージされた PR はありません
環境構築
- Dockerfile・Docker Compose ファイルなし
- プルリクエストのテンプレートなし
- コントリビューションガイドを読む
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
electric-sql/electric のほかの issue
-
camelToSnake and snakeCamelMapper() drop the underscore when a digit precedes a camelCase boundaryオープン
難易度 2/5 1〜3時間 初心者へのやさしさ 85/100
electric-sql/electric#4855 ·
-
ShapeStream does not notify subscribers when it gives up on a non-Error value対応中かも @Tyagiquamar が 19 日前に担当しました。 オープン
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
electric-sql/electric#4830 ·
-
@electric-sql/client throws SecurityError on import when the browser blocks site storage (typeof localStorage outside try)対応中かも @mohanedmoh が 37 日前に担当しました。 オープン
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
electric-sql/electric#4826 · コメント 1 件 ·
-
agents-chat-starter: spawn PUT goes to the durable-streams proxy, so no entity is created and agents never wake対応中かも @adityavkk が 133 日前に担当しました。 オープン
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
electric-sql/electric#4687 ·
-
Listen IP addressオープンfeature request triage
難易度 2/5 1〜3時間 初心者へのやさしさ 68/100
electric-sql/electric#4150 · コメント 1 件 ·
electric-sql/electric の issue をすべて見る
似ている issue
-
clawsweeper:fix-shape-clear clawsweeper:queueable-fix clawsweeper:source-repro impact:ux-friction issue-rating: 🦞 diamond lobster no-stale P2
難易度 2/5 1〜3時間 初心者へのやさしさ 68/100
openclaw/openclaw#199885 · コメント 2 件 · リアクション 1 件 ·
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 88/100
-
feature needs-triage
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
BoardGamesEmpire/board-games-empire-backend#1306 ·
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 80/100
sequant-io/sequant#1689 ·
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 80/100