Overlay upper: support virtio-blk scratch or tmpfs end to end, and align docs

Cerrado
#558 2 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

@ppenna ya está trabajando en esto.

Desde el 28/10/2026.

  • #977 de @ppenna — abierto

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
8/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
linux, python, rust, shell

Línea de trabajo

Start with the guest mount logic in guest/common/nvx-init-agent (around lines 482-499, where it requires a scratch device) and the launcher in scripts/nvx_tools/sandbox.py (around lines 561-612). Then read OpenVMM's ramfs_overlay_requested in openvmm/openvmm_defs/src/microvm.rs. Done means one agreed token and name for the RAM upper, a guest that mounts a size-bounded tmpfs, and smoke tests in guest/common/nvx-sandbox-smoke and scripts/test_nvx_tools.py that check the declared upper. Settle the open naming and sizing questions with maintainers first.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Goal

Make the implementation and the docs agree on this statement:

For the read-write top of the overlay, NVX supports either a scratch space (a virtio-blk device) or a tmpfs in RAM.

Today only the scratch half is true end to end. OpenVMM accepts a RAM-backed overlay layout, but no NVX guest, launcher, or test uses it, and the docs say the guest agents always use scratch. Found while reviewing the architecture guide in #529.

Current state

Code links are pinned to dev at 19a359b, design-guide links to the #529 head at 1574629, and OpenVMM links to the pinned gitlink nanvix/openvmm@7f225e2.

Implementation
  • Guest. nvx-init-agent stops with a scratch device is required, accepts only ext4, and mounts the overlay with upperdir=$scratch/upper. It has no tmpfs or ramfs path.
  • Launchers. sandbox.py requires an ext4 scratch image and always passes a scratch block and nvx_scratch=...,ext4. Nothing in scripts/, nvx-rs, or aci_edge_sandboxes emits nvx_overlay_upper.
  • VMM. OpenVMM's ramfs_overlay_requested accepts exactly one nvx_overlay_upper=ramfs token. The token only relaxes block validation: exactly one read-only distro block, no scratch, and no snapshot or restore. OpenVMM doesn't control what the guest mounts.
  • Tests. nvx-sandbox-smoke fails when the upper isn't /run/nvx/scratch/upper, and test_nvx_tools.py pins that check. OpenVMM unit-tests the token, but no NVX microVM scenario boots the RAM layout.
Docs
  • Design guide (#529). 4.1 says "The guest agents always keep the overlay upper layer on scratch, so the machine's RAM-backed overlay layout, which has no scratch, serves only guests that declare it." Table 4.1 and Figure 4.1 show only scratch, and the 1.1 goals list only scratch. 2.1, 3.4, and 5.1 describe the RAM layout as a VMM allowance and call it ramfs or "in RAM", never tmpfs.
  • Old doc/design/, which #529 removed. cold-boot.md says "The NVX init agent does not use this layout." sandbox-filesystem-and-agent-architecture.md argues against tmpfs for the workload: "Runtime tmpfs is still appropriate for small agent state, not the workload's general writable layer."

Related work

  • #535 tracks an image-backed runtime with "a tmpfs or ext4-scratch writable root".
  • #539 plans Upper::Tmpfs and Upper::Ext4Scratch in the nvx-rs guest core, and asks not to change the existing shell profile before its replacement is accepted.
  • #541 requires both upper modes on the standalone cold-start path.
  • #542 plans a blockless RAM-upper platform template.

This issue tracks what those share: one name and token for the RAM mode, launchers and tests that exercise it, and docs that describe what ships. If the guest side lands only through #539 and #541, close this issue once they land and the docs match.

Proposed work

  1. Name and token. Settle tmpfs versus ramfs. tmpfs takes a size= limit and fails writes with ENOSPC when full; ramfs has no limit and can grow until the guest runs out of memory. Either change the OpenVMM token (for example, to nvx_overlay_upper=tmpfs) in nanvix/openvmm and promote the pin, or keep ramfs as the layout name and document that the guest mounts tmpfs. The layout rejects capture and restore, so no snapshot records the token today.
  2. Guest. When the host declares the RAM layout, mount a size-bounded tmpfs with nosuid,nodev for the upper and work directories instead of resolving scratch, and unmount it at teardown. Validate the overlay options NVX uses (metacopy=on, xino=on) on that upper with the pinned kernel. Refuse a RAM layout that also attaches scratch, and a scratch layout without scratch.
  3. Launchers. Let nvx.py sandbox select the upper. In RAM mode, emit the token, attach no scratch block, accept only the distro layer, and reject snapshot options, matching OpenVMM. Do the same for nvx-rs through #541.
  4. Resources. A tmpfs upper consumes guest RAM, and its pages are charged to the memory cgroup of the task that writes them, so workload writes count against nvx_memory_max. Choose the default size and who sets it, and document the trade-off: scratch fails with ENOSPC on host storage, while a tmpfs upper adds guest memory pressure.
  5. Tests. Add a microVM scenario that boots one read-only distro block with a tmpfs upper and no scratch. Make nvx-sandbox-smoke and its test_nvx_tools.py vectors check the declared upper instead of scratch only. Add negative cases for mismatched layouts and for snapshot options in RAM mode.
  6. Docs. Describe both modes, their restrictions, and their trade-offs, and remove the "always on scratch" statements:
    • The architecture guide in doc/design/: 1.1, 2.1.3, 2.5 Table 2.7, 3.4 (intro and Table 3.11), 4.1 (Table 4.1, Figure 4.1, and the 4.1.1 assembly and teardown steps), 4.3 (memory accounting), and 5.1.2.
    • doc/run.md and doc/usage.md for the launcher option, and the OpenVMM Guide's run.md and cli.md if the token changes.

Acceptance criteria

  • An NVX guest boots a layered sandbox with either an ext4 scratch upper on virtio-blk or a size-bounded tmpfs upper, chosen by the host, on Linux/KVM, Linux/MSHV, and Windows/WHP.
  • OpenVMM, the launchers, the guest, and the docs use one name and one token for the RAM mode.
  • At least one NVX launcher can request each mode, and the RAM mode attaches no scratch device.
  • Smoke checks and unit tests accept the declared upper and reject a mismatch.
  • The architecture guide in doc/design/ describes both modes, the RAM mode's restrictions (one read-only distro block, no capture or restore), and their resource trade-offs.

Open questions

  • Keep ramfs as the token value, switch to tmpfs, or accept both?
  • Should guest support land first in the shell nvx-init-agent, or only in the nvx-rs guest core from #539?
  • What is the default tmpfs size, and is it a host token or a fraction of guest RAM?
  • Should a tmpfs upper share the workload's memory limit or get its own?

Out of scope

  • Capturing or restoring a populated tmpfs upper. The #542 RAM-upper template is captured before overlay assembly.
  • A separate mismatch: the architecture guide describes one read-only lower layer (distro) and calls 0xd0004000..0xd0005fff unused, while OpenVMM and nvx-init-agent still define runtime and custom layers there. #556 tracks it.
Lenguaje dominante
Rust
Estrellas
758
Forks
48
Merge medio
16 h 57 min
PR fusionados (30 d)
441

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de microsoft/nvx

Todos los issues de microsoft/nvx

Issues similares

Más issues de Rust

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.