P4 epic: next experiments after the metal STOP — resume/destroy, throughput scaling, model-latency validation

Abierto
  • ¿Algo no está bien?
#3 3 comentarios 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
35/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
shell
Área
performance

Línea de trabajo

Start with the linked post-metal readout, PR rossoctl/moca#251, and the “Performance rungs” section of deploy/microvm/EXPERIMENTS.md; the measurements and rationale here frame the follow-on work. The issue lists seven independent experiments, each linked as a sub-issue, rather than specifying one implementation task. Choose a sub-issue and follow its experiment scope; done means reporting the measurement or validation that sub-issue calls for.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Goal

A secure sandbox service for multiple users and multiple sessions, with a sandbox created per request, optimized for throughput. Latency on the sandbox request path should not add significant overhead over a model's own "thinking time" between tool calls (typically hundreds of ms). The original spec target was ≤15 ms; the metal results below suggest we can likely relax that to something like ≤60 ms, but that number itself needs validating against real model latency rather than assumed.

Where this comes from

PR rossoctl/moca#251 ran the first bare-metal E10/E11 measurements on srv-r16b14s16 (2026-09-15). Headline results:

  • Warm hot path: 52.74 ms measured (E10 rung 2), against the original 15 ms bar — the design's own §7.2 stopping rule reads STOP.
  • Decomposition: acquire 0.00 ms · resume 28.58 ms · run 2.85 ms · destroy 21.31 ms · total 55.32 ms. Resume + destroy are 49.9 of the 55.3 ms — pre-building (replenishment, 10.98 ms CPU/restore) is not the problem and came in under its own estimate.
  • E11 density: knee at c=8 on both arms (Firecracker microVM and container), bound: replenishment. Throughput climbs to ~39/s (microVM) / ~55/s (container) at c=8, "peaks" at c=16 only by riding degraded latency/cold-acquire rate, then declines at c=32/64 — a congestion collapse, not a ceiling to size past.
  • Host resource headroom is enormous at every rung tested: hostCpuFraction ~0.001 flat, Σ PSS ≤ 0.41 GB at 128 resident microVMs (CoW sharing against the pinned golden-snapshot memfile — see remote-worker/internal/vmpool/pin_linux.go). Nothing resembling a CPU/memory ceiling was reached.
  • Prediction scoreboard (deploy/microvm/predictions.json, sealed before the first rung): 2 supported, 0 falsified, 2 inconclusive, 1 not evaluable.
  • Full detail: deploy/microvm/EXPERIMENTS.md ("Performance rungs" section).

Why relax 15 ms → ~60 ms at all

The 15 ms bar was set before any restore/destroy cost was measured, and assumed building could be hidden entirely behind model think-time. It's aggressive relative to a "hundreds of ms" thinking budget, and 52.74 ms already sits within 2x of this host's own container baseline (41 ms — sealed prediction 2: supported). But relaxing it is only justified if real model latency actually dominates the way the original design assumed — which has never been measured (E11 drives synthetic Execs, not real agent turns).

Sub-issues

This epic tracks seven follow-on experiments, roughly in priority order against the stated goal. Each is scoped to stand on its own — pick any one and turn it into a spec without needing to read the others first.

  • rossoctl/moca-experiments#4 — Measure real model think-time distribution
  • rossoctl/moca#258 — Decompose resume + destroy into their sub-costs
  • rossoctl/moca#259 — Worker-process parallelism: does N workers give N× throughput?
  • rossoctl/moca#260 — Standby-depth / replenishment-concurrency sweep
  • rossoctl/moca#266 — Repeat the E10/E11 ladders on nested virtualization for a metal/nested ratio
  • rossoctl/moca#261 — E11 with a real model in the loop
  • rossoctl/moca-experiments#6 — Score sealed prediction 1 (memory/process ceiling)

Explicitly out of scope for this epic

  • Reusing/parking a VM across multiple Execs within one run to dodge resume/destroy. E10 already shows parked-vs-fresh-child resume times are nearly identical (28.58 vs 28.56 ms), so any parked-shell saving would show up in run-time, not resume — and pursuing it would mean giving up per-Exec isolation, which conflicts with "sandbox created per request" as currently scoped.
  • Reviving Cloud Hypervisor investment. It stops at 3/10 correctness gates and cannot complete a restore with virtio-fs attached (see the correctness-gates section of deploy/microvm/EXPERIMENTS.md). Standing recommendation: keep the code, stop investing, and let the one-spare-per-job number decide it later.

Context: opened from the post-metal readout discussion on PR rossoctl/moca#251 (2026-09-15 bare-metal run, srv-r16b14s16). See also rossoctl/moca#255 (metal-run cleanup follow-ups — unrelated to this epic's numbers).

Lenguaje dominante
Shell
Estrellas
0
Forks
1
Merge medio
6 h 16 min
PR fusionados (30 d)
7

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de rossoctl/moca-experiments

Todos los issues de rossoctl/moca-experiments

Issues similares

Más issues de Shell/Bash

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.