bug: gator supervisor retries transient failures forever despite a documented cap
@purp 已经在做这个了。
开始于 2026年9月14日。
评估
这个 Issue 还没有评估数据。
描述
User Story
As a maintainer supervising a gator agent, I want the supervisor to stop after repeated transient failures, so that a wedged or permanently-failing gator surfaces for human attention instead of burning sandbox time indefinitely.
Problem Statement
OPENSHELL_AGENT_MAX_TRANSIENT_FAILURES (default 5) is documented as a retry cap, but it does not cap anything. In scripts/agents/runtime/supervisor.sh, retry_watch_cycle() uses the value only to choose between two nearly identical log messages via a modulo test; both branches fall through to sleep_with_heartbeat and return, and the watch loop continues unconditionally. There is no exit path on repeated transient failures.
The operator-facing documentation states the opposite. .agents/skills/launch-openshell-gator/SKILL.md:313 tells the operator to "Let supervisor retry unless repeated failures hit the configured cap," which describes behavior that does not exist.
Impact / Why This Matters
A gator hitting a persistent auth, transport, or infrastructure failure is classified as transient_failure and retries forever. Because the supervisor never exits, the sandbox stays alive and no terminal_failure sentinel is ever written, so the documented operator signal for "this needs a human" never fires.
The current workaround is for an operator to notice the condition by reading logs and delete the sandbox manually. That is insufficient because the sentinel table is the documented mechanism for deciding whether human action is needed, and it cannot report a state the supervisor cannot reach. Nothing else bounds the run: there is no timeout or cleanup path elsewhere in the gator lifecycle, so an unattended failure is only ever found by someone going to look.
Acceptance Criteria
- With
OPENSHELL_AGENT_MAX_TRANSIENT_FAILURES=N, the supervisor stops retrying after N consecutive transient failures rather than continuing indefinitely. - On hitting the cap, the supervisor emits a
terminal_failuresentinel with a reason identifying the transient failure, matching the sentinel contract the skill documents. - Consecutive-failure counting resets after a successful cycle, so intermittent failures spread over a long run do not accumulate to the cap.
-
OPENSHELL_AGENT_MAX_TRANSIENT_FAILURES=0retains its existing meaning of unlimited retries, and that meaning is documented. -
.agents/skills/launch-openshell-gator/SKILL.mddescribes the actual behavior, including what the operator should do when the cap is hit.
Reproduction Steps
- Launch a gator with a low cap:
OPENSHELL_AGENT_MAX_TRANSIENT_FAILURES=2 ./scripts/agents/run.sh --agent gator ... - Induce a repeatable transient failure, for example by invalidating the
ghcredential the sandbox uses so watch cycles fail on transport/auth. - Watch the supervisor log.
Observed: openshell-agent: transient watch failure N (<reason>); retrying in Ns repeats without bound; the wording alternates at multiples of the cap but the loop never exits and no terminal_failure sentinel is written.
Expected: the supervisor stops after 2 consecutive transient failures and reports status=terminal_failure.
Environment
- OpenShell: repository scripts at
5b9daab93(not a released artifact) - OS: macOS 15 (Darwin 25.6.0); the supervisor is plain bash and the defect is host-independent
- Component:
scripts/agents/runtime/supervisor.sh, gator agent runtime
Logs
scripts/agents/runtime/supervisor.sh:21 MAX_TRANSIENT_FAILURES="${OPENSHELL_AGENT_MAX_TRANSIENT_FAILURES:-5}"
scripts/agents/runtime/supervisor.sh:140 if [[ "$MAX_TRANSIENT_FAILURES" -gt 0 ]]; then
scripts/agents/runtime/supervisor.sh:141 if [[ $((transient_failures % MAX_TRANSIENT_FAILURES)) -eq 0 ]]; then
# both branches only echo, then sleep and return
- 主要语言
- Rust
- 星标
- 8.7k
- 派生
- 1.3k
- 平均合并
- 2 天 6 小时
- 30 天内合并 PR
- 297
贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
NVIDIA/OpenShell 的其他 Issue
-
area:docs
难度 1/5 1 小时以内 新手友好度 88/100
-
state:triage-needed
难度 2/5 1-3 小时 新手友好度 82/100
-
area:cli state:validated
难度 2/5 1-3 小时 新手友好度 72/100
-
state:triage-needed
难度 1/5 1 小时以内 新手友好度 90/100
-
area:build spike state:review-ready state:stale
难度 2/5 半天 新手友好度 68/100
相似的 Issue
-
bug github_actions
难度 2/5 1-3 小时 新手友好度 75/100
registrystack/registry-stack#1393 ·
-
难度 2/5 1-3 小时 新手友好度 75/100
longbridge/gpui-kit#3223 ·
-
bug engine
难度 2/5 1-3 小时 新手友好度 65/100
rocky-data/rocky#2181 ·
-
难度 2/5 1-3 小时 新手友好度 70/100
oasisprotocol/oasis-sdk#2523 ·
-
bot:ai-assisted component:indexer QA-roadmap status:untriaged
难度 2/5 1-3 小时 新手友好度 75/100
midnightntwrk/midnight-indexer#1557 ·