buzz-acp: idle-pool teardown race silently swallows a concurrently arriving message (no log trace beyond routine steer ack)
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 48/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- rust
- Domain
- backend, distributed-systems
Research direction
Read idle_pool_sleep_due in crates/buzz-acp/src/lib.rs:2164-2179, then trace the reaper select! arm at lib.rs:3606-3654 through shutdown_agent_pool and the SteerAck branches at lib.rs:3889-3922. Reproduce a message arriving as the 900-second teardown begins; done means it is deferred, dispatched to the fresh pool, re-queued, or surfaced as an error rather than silently dropped.
Written by the indexing model from the issue text.
Description
Summary
buzz-acp's idle-pool reaper (idle_pool_sleep_due, crates/buzz-acp/src/lib.rs:2164-2179, invoked from the reaper-tick select! arm at lib.rs:3606-3654) tears the agent pool back to a lazy/listening state after idle_pool_sleep_secs (900s in our config) of inactivity via shutdown_agent_pool. When a new message/wake arrives at, or just before, the same tick that triggers this teardown, the message can be silently dropped: no error, no retry record, nothing but a routine tracing::info! line —
crates/buzz-acp/src/lib.rs:3914-3922
tracing::info!(
channel = %channel_id,
event_id = %event_id,
?ack,
release_withheld,
drop_withheld,
signal_fallback,
"non-cancelling steer ack received"
);
— which reads as routine steer bookkeeping, not as a dropped-message signal.
This looks like a distinct failure mode from the already-tracked #6378 / #7171. Those describe last_activity not being refreshed for genuinely in-flight work (background subagents, long turns), so the pool is torn down while the agent is still busy. Here, by contrast, the pool really is idle and due for teardown — the bug is that a message arriving concurrently with that teardown races shutdown_agent_pool / pool rebuild and is lost instead of being deferred, dispatched to the fresh pool, or re-queued.
Scope
Affects every buzz-acp managed agent, independent of subscribe mode (Mentions vs Config). Mentions-mode agents hit it far more often in practice per-wake — with fewer incidental messages to keep the pool warm between targeted wakes, the pool has usually gone fully idle and been torn down by the time the next @mention/wake lands, so the race window is entered more often relative to total wake attempts.
Evidence
The "non-cancelling steer ack received" log line is the direct fingerprint of a steer racing pool teardown/rebuild. Counts observed in our deployment's agent logs:
harness-dev-orch: 69 occurrences (highest raw count, but least visible impact — many subscribed channels mean other traffic quickly re-wakes the pool, masking the drop)stock_orch: 8 occurrencesstock-news: 5 occurrences
Real-world impact
stock-news's pre-market briefing pipeline depends on a cron-triggered T-30 wake message. On 2026-09-16, 09-17, and 09-18, this wake was accepted by the relay (accepted:true) but the agent never responded to it — 3 relay retries, then retry-exhausted — consistent with the wake landing inside an idle-pool teardown window and being swallowed.
Suggested repro / investigation
Dispatch a message exactly at/after idle_pool_sleep_bound elapses (900s default) while pool_ready is true and the idle_pool_sleep_due gates (work_queued, prompt_tasks_in_flight, wake_or_respawn_in_flight) are all clear, and trace what happens to that message through shutdown_agent_pool and pool rebuild (lib.rs:3635-3653). The race window looks like it's between the gate check passing and the teardown/rebuild actually completing, rather than in the gate logic itself — i.e. a message dispatched into the old pool concurrently with (or immediately after) shutdown_agent_pool gets a steer ack against a pool that's mid-teardown, hits one of the non-fallback SteerAck branches (lib.rs:3889-3913), and is neither retried against the new pool nor surfaced as an error.
Related issues
- #6378 — idle pool teardown kills an in-flight session, losing a queued background-task wakeup. Related mechanism (same reaper), different trigger: in-flight session activity not counted as activity, vs. a new message racing the teardown of an already-idle pool.
- #7171 — idle-pool reaper tears down right after a long turn completes. Same reaper code path, different root cause: stale
last_activitynot refreshed on turn completion, vs. a delivery race during an already-correctly-triggered teardown.
Neither of the above describes this specific scenario — a message that arrives concurrently with an already-due idle teardown being silently swallowed with no log trace beyond a routine steer-ack info line — so filing separately rather than piling onto either.
- Dominant language
- Rust
- Stars
- 33.7k
- Forks
- 4.4k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 239
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from block/buzz
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
-
workflow_sink's mention parser never masks code regions — @name inside a code span wakes the agent Open
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 Half a day Newbie friendliness 88/100
-
Difficulty 1/5 Under an hour Newbie friendliness 92/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Eynzof/Hermes-CN-Desktop#610 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
gitbutlerapp/gitbutler#15998 · 1 comment ·
-
bug triage:deciding
Difficulty 1/5 Under an hour Newbie friendliness 88/100
open-telemetry/otel-arrow#4132 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100