feat(supervisor): hold reconnect while a sandbox is suspended by its backend
Les mainteneurs répondent en général sous 1 jour
Personne n'a encore pris cette issue.
Évaluation
- Difficulté
- 5/5
- Temps estimé
- Plus d'une semaine
- Accessibilité débutants
- 35/100
Piste de recherche
Start by reading the supervisor reconnect logic and the backend suspend/resume contract added in draft PR #3898; the issue does not name files or tests. The key design question is how the supervisor receives suspend and resume signals, including in the multi-tenant case. Done means suspended sandboxes trigger no connection attempts, resume restores the session without restarting the workload, and ordinary connection loss still reconnects.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Description
User Story
As an operator running OpenShell sandboxes on a compute backend that suspends idle sandboxes,
I want the supervisor to hold its reconnect while a sandbox is suspended,
so that idle sandboxes stay suspended and stop using compute until they are needed.
Problem Statement
The supervisor treats any drop of its connection to the sandbox as a failure and reconnects right away. When a backend suspends a sandbox, that connection drops too. On a backend where a connection to a suspended sandbox resumes it, such as Agent Substrate, the supervisor's reconnect wakes the sandbox straight back up, so it never stays suspended.
The supervisor has no way to tell "the sandbox was suspended" from "the connection broke".
Impact / Why This Matters
Suspend and resume on a backend only saves compute if a suspended sandbox stays suspended. Today every suspend is undone by the supervisor's own reconnect.
We saw this in a prototype on Agent Substrate: OpenShell's sandbox runtime (at ec49209d) inside a Substrate microVM, with OpenShell's own supervisor reaching it through the Substrate ingress.
- Without any change, suspending the sandbox closed the connection, the supervisor redialed, and the redial resumed the sandbox shortly after the suspend.
- With a stand-in component that held the supervisor's redial until the sandbox resumed, a 15-minute suspend stayed suspended and resumed cleanly on a different machine. The runtime recovered the session after the redial, since its reconnect timeout only starts counting after resume.
The same question applies to the planned multi-tenant supervisor: if it keeps a connection open for every suspended sandbox, suspended sandboxes still get woken and still cost the supervisor a connection each.
Proposed Design
- When a backend suspends a sandbox, the supervisor learns that the sandbox is suspended, and does not reconnect to it while it stays suspended.
- When the sandbox resumes, the supervisor reconnects and the session continues, without restarting the workload.
- An unexpected connection loss, with no suspend, still triggers a reconnect as it does today.
- This works the same for today's per-sandbox supervisor and for a multi-tenant supervisor.
Draft PR #3898 (delegated backend wire contract) already adds suspend and resume to the backend contract, so the gateway knows when a sandbox is suspended. That may be the natural path for the supervisor to learn about it too. We'd defer to you on the right signal.
Acceptance Criteria
- A sandbox suspended by its backend stays suspended until it is resumed. The supervisor makes no connection attempts to it in the meantime.
- After resume, the supervisor re-establishes its session and the workload continues without a restart.
- A connection loss that is not a suspend still triggers a reconnect, as today.
- The behavior is the same with a multi-tenant supervisor, and a suspended sandbox does not hold a supervisor connection.
Alternatives Considered
- The backend refuses connections to a suspended sandbox: the redials then fail, and in our runs the supervisor exited after a refusal.
- A component next to the supervisor holds its redial while the sandbox is suspended: this is what our prototype did, and it works, but every backend would have to build one, and it has to work out state the gateway already knows.
- The runtime drops its old connection when the same supervisor redials: useful on its own, but it doesn't stop the redial from waking the sandbox.
Checklist
- I've reviewed existing issues and the published docs
- This is a design proposal, not a "please build this" request
@drew, related to the suspend and resume work in #3898. Happy to share more detail from the prototype.
- Langage dominant
- Rust
- Étoiles
- 15.4k
- Forks
- 1.7k
- Merge moyen
- 1 j 19 h
- PR mergées (30 j)
- 347
Préparer son environnement
- Aucun Dockerfile ni fichier Docker Compose
- Propose un modèle de pull request
- Lire le guide de contribution
Par où commencer
- Lisez l'issue en entier, puis le guide de contribution du projet.
- Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
- Forkez le dépôt et travaillez sur une branche.
- Ouvrez une pull request qui référence le numéro de l'issue.
Autres issues de NVIDIA/OpenShell
-
state:triage-needed
Difficulté 2/5 1-3 heures Accessibilité débutants 65/100
Les mainteneurs répondent en général sous 1 jour
-
state:triage-needed
Difficulté 2/5 1-3 heures Accessibilité débutants 70/100
Les mainteneurs répondent en général sous 1 jour
-
docs: document workspace and provider label capabilitiesPeut-être pris @johntmyers l’a pris il y a 5 jours. Ouvertearea:docs
Difficulté 2/5 1-3 heures Accessibilité débutants 72/100
NVIDIA/OpenShell#4250 · 2 commentaires ·
Les mainteneurs répondent en général sous 1 jour
-
bug(driver-mxc): test helper fails to compile after gateway-name argumentPeut-être pris @feloy l’a pris il y a 6 jours. Ouvertestate:triage-needed
Difficulté 1/5 Moins d'une heure Accessibilité débutants 88/100
Les mainteneurs répondent en général sous 1 jour
-
bug: install.sh ignores XDG_CONFIG_HOME for the local gateway configPeut-être pris @fede-kamel l’a pris il y a 10 jours. Ouvertearea:cli os:linux os:macos state:validated
Difficulté 2/5 1-3 heures Accessibilité débutants 88/100
NVIDIA/OpenShell#4042 · 2 commentaires ·
Les mainteneurs répondent en général sous 1 jour
Toutes les issues de NVIDIA/OpenShell
Issues similaires
-
llm translation
Difficulté 2/5 1-3 heures Accessibilité débutants 76/100
Les mainteneurs répondent en général sous 1 jour
-
Difficulté 2/5 1-3 heures Accessibilité débutants 77/100
CybercentreCanada/sawp#13 ·
-
area: backend type: enhancement
Difficulté 2/5 1-3 heures Accessibilité débutants 61/100
armadavalor/WinKnife#6 ·
-
Difficulté 2/5 1-3 heures Accessibilité débutants 68/100
-
Difficulté 2/5 1-3 heures Accessibilité débutants 78/100
5omeOtherGuy/phaseone#870 ·
Les mainteneurs répondent en général sous 1 jour