feat(supervisor): hold reconnect while a sandbox is suspended by its backend
Los mantenedores suelen responder en 1 día
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 5/5
- Tiempo estimado
- Más de una semana
- Aptitud para principiantes
- 35/100
Línea de trabajo
Start by reading the supervisor reconnect logic and the backend suspend/resume contract added in draft PR #3898; the issue does not name files or tests. The key design question is how the supervisor receives suspend and resume signals, including in the multi-tenant case. Done means suspended sandboxes trigger no connection attempts, resume restores the session without restarting the workload, and ordinary connection loss still reconnects.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
User Story
As an operator running OpenShell sandboxes on a compute backend that suspends idle sandboxes,
I want the supervisor to hold its reconnect while a sandbox is suspended,
so that idle sandboxes stay suspended and stop using compute until they are needed.
Problem Statement
The supervisor treats any drop of its connection to the sandbox as a failure and reconnects right away. When a backend suspends a sandbox, that connection drops too. On a backend where a connection to a suspended sandbox resumes it, such as Agent Substrate, the supervisor's reconnect wakes the sandbox straight back up, so it never stays suspended.
The supervisor has no way to tell "the sandbox was suspended" from "the connection broke".
Impact / Why This Matters
Suspend and resume on a backend only saves compute if a suspended sandbox stays suspended. Today every suspend is undone by the supervisor's own reconnect.
We saw this in a prototype on Agent Substrate: OpenShell's sandbox runtime (at ec49209d) inside a Substrate microVM, with OpenShell's own supervisor reaching it through the Substrate ingress.
- Without any change, suspending the sandbox closed the connection, the supervisor redialed, and the redial resumed the sandbox shortly after the suspend.
- With a stand-in component that held the supervisor's redial until the sandbox resumed, a 15-minute suspend stayed suspended and resumed cleanly on a different machine. The runtime recovered the session after the redial, since its reconnect timeout only starts counting after resume.
The same question applies to the planned multi-tenant supervisor: if it keeps a connection open for every suspended sandbox, suspended sandboxes still get woken and still cost the supervisor a connection each.
Proposed Design
- When a backend suspends a sandbox, the supervisor learns that the sandbox is suspended, and does not reconnect to it while it stays suspended.
- When the sandbox resumes, the supervisor reconnects and the session continues, without restarting the workload.
- An unexpected connection loss, with no suspend, still triggers a reconnect as it does today.
- This works the same for today's per-sandbox supervisor and for a multi-tenant supervisor.
Draft PR #3898 (delegated backend wire contract) already adds suspend and resume to the backend contract, so the gateway knows when a sandbox is suspended. That may be the natural path for the supervisor to learn about it too. We'd defer to you on the right signal.
Acceptance Criteria
- A sandbox suspended by its backend stays suspended until it is resumed. The supervisor makes no connection attempts to it in the meantime.
- After resume, the supervisor re-establishes its session and the workload continues without a restart.
- A connection loss that is not a suspend still triggers a reconnect, as today.
- The behavior is the same with a multi-tenant supervisor, and a suspended sandbox does not hold a supervisor connection.
Alternatives Considered
- The backend refuses connections to a suspended sandbox: the redials then fail, and in our runs the supervisor exited after a refusal.
- A component next to the supervisor holds its redial while the sandbox is suspended: this is what our prototype did, and it works, but every backend would have to build one, and it has to work out state the gateway already knows.
- The runtime drops its old connection when the same supervisor redials: useful on its own, but it doesn't stop the redial from waking the sandbox.
Checklist
- I've reviewed existing issues and the published docs
- This is a design proposal, not a "please build this" request
@drew, related to the suspend and resume work in #3898. Happy to share more detail from the prototype.
- Lenguaje dominante
- Rust
- Estrellas
- 15.4k
- Forks
- 1.7k
- Merge medio
- 1 d 19 h
- PR fusionados (30 d)
- 347
Preparar el entorno
- Sin Dockerfile ni archivo de Docker Compose
- Tiene una plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de NVIDIA/OpenShell
-
state:triage-needed
Dificultad 2/5 1-3 horas Aptitud para principiantes 65/100
Los mantenedores suelen responder en 1 día
-
state:triage-needed
Dificultad 2/5 1-3 horas Aptitud para principiantes 70/100
Los mantenedores suelen responder en 1 día
-
docs: document workspace and provider label capabilitiesPosiblemente ocupada @johntmyers la tomó hace 5 días. Abiertoarea:docs
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
NVIDIA/OpenShell#4250 · 2 comentarios ·
Los mantenedores suelen responder en 1 día
-
bug(driver-mxc): test helper fails to compile after gateway-name argumentPosiblemente ocupada @feloy la tomó hace 7 días. Abiertostate:triage-needed
Dificultad 1/5 Menos de una hora Aptitud para principiantes 88/100
Los mantenedores suelen responder en 1 día
-
bug: install.sh ignores XDG_CONFIG_HOME for the local gateway configPosiblemente ocupada @fede-kamel la tomó hace 10 días. Abiertoarea:cli os:linux os:macos state:validated
Dificultad 2/5 1-3 horas Aptitud para principiantes 88/100
NVIDIA/OpenShell#4042 · 2 comentarios ·
Los mantenedores suelen responder en 1 día
Todos los issues de NVIDIA/OpenShell
Issues similares
-
[Bug]:设置的关于页面,“系统信息”部分排版异常Abiertobug
Dificultad 2/5 1-3 horas Aptitud para principiantes 63/100
metasequoiaime/msime#7328 ·
Los mantenedores suelen responder en 1 día
-
bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
eclipse-opensovd/classic-diagnostic-adapter#616 ·
Los mantenedores suelen responder en 2 días
-
documentation
Dificultad 1/5 Menos de una hora Aptitud para principiantes 86/100
Los mantenedores suelen responder en 1 día
-
track: content
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
0xAndersen/witness-log#10 ·
-
follow-up main-defect wave-A
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
carrick-sh/carrick#249 ·
Los mantenedores suelen responder en 1 día