feat(supervisor): hold reconnect while a sandbox is suspended by its backend
維護者通常 1 天內回覆
還沒有人認領這個 Issue。
評估
研究方向
Start by reading the supervisor reconnect logic and the backend suspend/resume contract added in draft PR #3898; the issue does not name files or tests. The key design question is how the supervisor receives suspend and resume signals, including in the multi-tenant case. Done means suspended sandboxes trigger no connection attempts, resume restores the session without restarting the workload, and ordinary connection loss still reconnects.
由索引模型根據 Issue 內容生成。
描述
User Story
As an operator running OpenShell sandboxes on a compute backend that suspends idle sandboxes,
I want the supervisor to hold its reconnect while a sandbox is suspended,
so that idle sandboxes stay suspended and stop using compute until they are needed.
Problem Statement
The supervisor treats any drop of its connection to the sandbox as a failure and reconnects right away. When a backend suspends a sandbox, that connection drops too. On a backend where a connection to a suspended sandbox resumes it, such as Agent Substrate, the supervisor's reconnect wakes the sandbox straight back up, so it never stays suspended.
The supervisor has no way to tell "the sandbox was suspended" from "the connection broke".
Impact / Why This Matters
Suspend and resume on a backend only saves compute if a suspended sandbox stays suspended. Today every suspend is undone by the supervisor's own reconnect.
We saw this in a prototype on Agent Substrate: OpenShell's sandbox runtime (at ec49209d) inside a Substrate microVM, with OpenShell's own supervisor reaching it through the Substrate ingress.
- Without any change, suspending the sandbox closed the connection, the supervisor redialed, and the redial resumed the sandbox shortly after the suspend.
- With a stand-in component that held the supervisor's redial until the sandbox resumed, a 15-minute suspend stayed suspended and resumed cleanly on a different machine. The runtime recovered the session after the redial, since its reconnect timeout only starts counting after resume.
The same question applies to the planned multi-tenant supervisor: if it keeps a connection open for every suspended sandbox, suspended sandboxes still get woken and still cost the supervisor a connection each.
Proposed Design
- When a backend suspends a sandbox, the supervisor learns that the sandbox is suspended, and does not reconnect to it while it stays suspended.
- When the sandbox resumes, the supervisor reconnects and the session continues, without restarting the workload.
- An unexpected connection loss, with no suspend, still triggers a reconnect as it does today.
- This works the same for today's per-sandbox supervisor and for a multi-tenant supervisor.
Draft PR #3898 (delegated backend wire contract) already adds suspend and resume to the backend contract, so the gateway knows when a sandbox is suspended. That may be the natural path for the supervisor to learn about it too. We'd defer to you on the right signal.
Acceptance Criteria
- A sandbox suspended by its backend stays suspended until it is resumed. The supervisor makes no connection attempts to it in the meantime.
- After resume, the supervisor re-establishes its session and the workload continues without a restart.
- A connection loss that is not a suspend still triggers a reconnect, as today.
- The behavior is the same with a multi-tenant supervisor, and a suspended sandbox does not hold a supervisor connection.
Alternatives Considered
- The backend refuses connections to a suspended sandbox: the redials then fail, and in our runs the supervisor exited after a refusal.
- A component next to the supervisor holds its redial while the sandbox is suspended: this is what our prototype did, and it works, but every backend would have to build one, and it has to work out state the gateway already knows.
- The runtime drops its old connection when the same supervisor redials: useful on its own, but it doesn't stop the redial from waking the sandbox.
Checklist
- I've reviewed existing issues and the published docs
- This is a design proposal, not a "please build this" request
@drew, related to the suspend and resume work in #3898. Happy to share more detail from the prototype.
- 主要語言
- Rust
- 星號
- 15.4k
- 分支
- 1.7k
- 平均合併
- 1 天 19 小時
- 30 天內合併 PR
- 347
環境準備
- 沒有 Dockerfile 或 Docker Compose 檔案
- 有 Pull Request 範本
- 閱讀貢獻指南
從這裡開始
- 先讀完整個 Issue,再讀專案的貢獻指南。
- 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
- Fork 儲存庫,在一個分支上完成修改。
- 送出 Pull Request,並在描述裡引用這個 Issue 編號。
NVIDIA/OpenShell 的其他 Issue
-
state:triage-needed
難度 2/5 1-3 小時 新手友好度 65/100
維護者通常 1 天內回覆
-
state:triage-needed
難度 2/5 1-3 小時 新手友好度 70/100
維護者通常 1 天內回覆
-
docs: document workspace and provider label capabilities可能已有人在做 @johntmyers 於 5 天前認領。 未關閉area:docs
難度 2/5 1-3 小時 新手友好度 72/100
NVIDIA/OpenShell#4250 · 2 則留言 ·
維護者通常 1 天內回覆
-
bug(driver-mxc): test helper fails to compile after gateway-name argument可能已有人在做 @feloy 於 7 天前認領。 未關閉state:triage-needed
難度 1/5 1 小時以內 新手友好度 88/100
維護者通常 1 天內回覆
-
bug: install.sh ignores XDG_CONFIG_HOME for the local gateway config可能已有人在做 @fede-kamel 於 11 天前認領。 未關閉area:cli os:linux os:macos state:validated
難度 2/5 1-3 小時 新手友好度 88/100
NVIDIA/OpenShell#4042 · 2 則留言 ·
維護者通常 1 天內回覆
相似的 Issue
-
ai-priority/P2 bug db/oracle user-priority/P1
難度 2/5 1-3 小時 新手友好度 67/100
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 70/100
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 72/100
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 76/100
swc-project/swc#12516 ·
維護者通常 1 天內回覆
-
app bug
難度 2/5 1-3 小時 新手友好度 76/100