garrytan/gbrain

Budget-cap denial retries in a tight loop with no backoff and no abort (20,276 retries in 23h)

オープン

#3,669 opened on 2026/07/31

 (2 件のコメント) (0 件のリアクション) (0 人の担当者)TypeScript (3,057 件のフォーク)github user discovery
fix-neededhelp wantedp1verified-real

Repository metrics

Stars
 (21,368 個のスター)
PR merge metrics
 (平均マージ 18d 10h) (30d で 162 merged PRs)

説明

Severity: high (pathological CPU/log burn; a cap makes things worse than no cap)

What happened. cycle.conversation_facts_backfill hit its per-run cost cap and then retried the same denied reservation continuously for ~23 hours instead of aborting the phase.

Verified counts — all 20,276 reserve_denied events sit in a single weekly ledger:

~/.gbrain/audit/budget-2026-W30.jsonl   6.2 MB   reserve_denied = 20276
(every other budget-*.jsonl and dream-budget-*.jsonl: 0)

Window: 2026-07-24T06:34:58Z2026-07-25T05:26:20Z~880 denials/hour, roughly one every 4 seconds, for 23 hours straight.

A representative event:

{
  "ts": "2026-07-24T06:34:58.942Z",
  "event": "reserve_denied",
  "label": "conversation_facts_backfill:brain-wide",
  "kind": "chat",
  "model": "anthropic:claude-sonnet-4-6",
  "sub_label": "gateway.chat",
  "projected_cost_usd": 0.06276,
  "cumulative_cost_usd": 0.44122,
  "max_cost_usd": 0.5
}

The arithmetic is stable and unwinnable: 0.44122 + 0.06276 = 0.50398 > 0.50. The projected cost never shrinks, so the reservation can never succeed — yet it is retried 20k times. The 6.2 MB ledger is itself a symptom.

Related design note. A cap set below one work unit's cost is strictly worse than no cap: the phase burns CPU and log volume forever while accomplishing zero work and spending $0.

Correction (2026-07-26, same session): an earlier draft of this file claimed cycle.propose_takes was in that same state — "halted at page 0, $0 spent." That was wrong and is retracted. The ledger shows propose_takes runs ~21–26×/day (autopilot is KeepAlive with ThrottleInterval 60), the cap is per run, not per day, and each run spends up to ~$0.048 before denial — ~$1.00–1.25/day, not $0. Its 292 submit_denied events are normal bounded cap behaviour, NOT the busy-loop described above. The 20,276-event busy-loop is conversation_facts_backfill only. Do not conflate the two phases.

Asks.

  1. On reserve_denied, abort the phase (or exponential-backoff) rather than retry immediately.
  2. Detect the unwinnable case at phase start: if max_cost_usd < projected_unit_cost, refuse to start with a clear message naming both numbers, instead of spinning.
  3. Rate-limit or dedupe identical denial events in the audit ledger.

コントリビューターガイド