P4 epic: next experiments after the metal STOP — resume/destroy, throughput scaling, model-latency validation

Aperta
  • Qualcosa non va?
#3 3 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
35/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
shell
Ambito
performance

Direzione di ricerca

Start with the linked post-metal readout, PR rossoctl/moca#251, and the “Performance rungs” section of deploy/microvm/EXPERIMENTS.md; the measurements and rationale here frame the follow-on work. The issue lists seven independent experiments, each linked as a sub-issue, rather than specifying one implementation task. Choose a sub-issue and follow its experiment scope; done means reporting the measurement or validation that sub-issue calls for.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Goal

A secure sandbox service for multiple users and multiple sessions, with a sandbox created per request, optimized for throughput. Latency on the sandbox request path should not add significant overhead over a model's own "thinking time" between tool calls (typically hundreds of ms). The original spec target was ≤15 ms; the metal results below suggest we can likely relax that to something like ≤60 ms, but that number itself needs validating against real model latency rather than assumed.

Where this comes from

PR rossoctl/moca#251 ran the first bare-metal E10/E11 measurements on srv-r16b14s16 (2026-09-15). Headline results:

  • Warm hot path: 52.74 ms measured (E10 rung 2), against the original 15 ms bar — the design's own §7.2 stopping rule reads STOP.
  • Decomposition: acquire 0.00 ms · resume 28.58 ms · run 2.85 ms · destroy 21.31 ms · total 55.32 ms. Resume + destroy are 49.9 of the 55.3 ms — pre-building (replenishment, 10.98 ms CPU/restore) is not the problem and came in under its own estimate.
  • E11 density: knee at c=8 on both arms (Firecracker microVM and container), bound: replenishment. Throughput climbs to ~39/s (microVM) / ~55/s (container) at c=8, "peaks" at c=16 only by riding degraded latency/cold-acquire rate, then declines at c=32/64 — a congestion collapse, not a ceiling to size past.
  • Host resource headroom is enormous at every rung tested: hostCpuFraction ~0.001 flat, Σ PSS ≤ 0.41 GB at 128 resident microVMs (CoW sharing against the pinned golden-snapshot memfile — see remote-worker/internal/vmpool/pin_linux.go). Nothing resembling a CPU/memory ceiling was reached.
  • Prediction scoreboard (deploy/microvm/predictions.json, sealed before the first rung): 2 supported, 0 falsified, 2 inconclusive, 1 not evaluable.
  • Full detail: deploy/microvm/EXPERIMENTS.md ("Performance rungs" section).

Why relax 15 ms → ~60 ms at all

The 15 ms bar was set before any restore/destroy cost was measured, and assumed building could be hidden entirely behind model think-time. It's aggressive relative to a "hundreds of ms" thinking budget, and 52.74 ms already sits within 2x of this host's own container baseline (41 ms — sealed prediction 2: supported). But relaxing it is only justified if real model latency actually dominates the way the original design assumed — which has never been measured (E11 drives synthetic Execs, not real agent turns).

Sub-issues

This epic tracks seven follow-on experiments, roughly in priority order against the stated goal. Each is scoped to stand on its own — pick any one and turn it into a spec without needing to read the others first.

  • rossoctl/moca-experiments#4 — Measure real model think-time distribution
  • rossoctl/moca#258 — Decompose resume + destroy into their sub-costs
  • rossoctl/moca#259 — Worker-process parallelism: does N workers give N× throughput?
  • rossoctl/moca#260 — Standby-depth / replenishment-concurrency sweep
  • rossoctl/moca#266 — Repeat the E10/E11 ladders on nested virtualization for a metal/nested ratio
  • rossoctl/moca#261 — E11 with a real model in the loop
  • rossoctl/moca-experiments#6 — Score sealed prediction 1 (memory/process ceiling)

Explicitly out of scope for this epic

  • Reusing/parking a VM across multiple Execs within one run to dodge resume/destroy. E10 already shows parked-vs-fresh-child resume times are nearly identical (28.58 vs 28.56 ms), so any parked-shell saving would show up in run-time, not resume — and pursuing it would mean giving up per-Exec isolation, which conflicts with "sandbox created per request" as currently scoped.
  • Reviving Cloud Hypervisor investment. It stops at 3/10 correctness gates and cannot complete a restore with virtio-fs attached (see the correctness-gates section of deploy/microvm/EXPERIMENTS.md). Standing recommendation: keep the code, stop investing, and let the one-spare-per-job number decide it later.

Context: opened from the post-metal readout discussion on PR rossoctl/moca#251 (2026-09-15 bare-metal run, srv-r16b14s16). See also rossoctl/moca#255 (metal-run cleanup follow-ups — unrelated to this epic's numbers).

Lingua principale
Shell
Stelle
0
Fork
1
Merge medio
6h 16m
PR unite (30g)
7

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di rossoctl/moca-experiments

Tutte le issue di rossoctl/moca-experiments

Issue simili

Altre issue su Shell/Bash

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.