E8: run VM density + saturation (both arms) and record the result

Aperta
  • Qualcosa non va?
#11 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
35/100
Tipo di issue
Documentazione
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
shell
Ambito
performance

Direzione di ricerca

First check the listed prerequisites and read the amended §5.2 in rossoctl/moca-experiments#10; decide whether the run is single- or mixed-subject before starting. Run and record the capacity and realism arms separately, including the specified metrics and knee attribution. Done means the results and scope are recorded in deploy/vm/EXPERIMENTS.md, with a pointer added to docs/experiment-results.md.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Summary

Run E8 — VM density and saturation (P6 §5.2) — and record the result. The spec amendment and driver
work are rossoctl/moca-experiments#10; this issue is the run itself, which rossoctl/moca-experiments#10 explicitly does not cover.

Splitting them is deliberate. rossoctl/moca-experiments#10 lands a two-arm methodology (capacity + realism) and the driver
changes that make a knee reachable; it can be reviewed and merged without any hardware booked. The run
consumes that methodology, produces the number, and is where a bound=not-observed outcome has to be
adjudicated rather than merged. Keeping them in one issue is what let the previous calibrated run be
recorded as a density figure when it was stub-sleep-bound.

Preconditions — all of them, checked before booking anything

# Gate Why it blocks
1 rossoctl/moca-experiments#10 merged Without the capacity arm the ladder tops out at the admission cap W×S=32, a config choice, and the run reproduces bound=not-observed
2 PR rossoctl/moca#253 merged /turn's tool calls must reach the sandbox pool. A run where tools execute locally measures a harness with no hands tier — and the counting probe is what catches it
3 rossoctl/moca#239 merged if mixed-subject rungs are claimed P6 §3.6/§7: single-subject rungs exercise every mechanism this slice adds and need no P5. Mixed subjects and the isolation claim need it. Decide which run this is before starting
4 Model stub deployed with a tool-call profile (§5.4) A text-only stub means no session reaches the sandbox and the density number silently excludes the hands tier
5 Duty basis pinned to one row of §2.3 Default E6/OCP: duty = 0.061–0.079, provision at 12.6–16.5. The driver asserts K ≥ ceil(W × S × duty) with duty and the ratio from the same row

Gate 3 is a scoping decision, not a dependency to wait on. Per P6 §3.6, P5 slipping delays a claim,
not the work — so a single-subject E8 is runnable the moment gates 1, 2, 4, 5 hold, and should not be
held behind rossoctl/moca#239.

What to run

Both arms from rossoctl/moca-experiments#10's amended §5.2, each recorded separately and never merged into one figure:

  • capacity arm — trivial exec, fast stub profile, sandbox tier non-binding and non-competing.
    Sweep offered concurrency upward with SH_WORKERS × SH_TURNS_PER_WORKER and
    KAGENTI_SANDBOX_CAP set well past the expected knee. Reports an upper bound on the harness
    tier     — explicitly not a deployable density.
  • realism arm — calibrated duty (e6-ocp basis), stub profile 300/12/64. This is the deployable
    number, sandbox-shaped by construction.

W=1 is a rung, not an aside: it makes "one process would be simpler" a data point (§5.2).

Knee detection reuses E6's sustained-decline detectKnee with degradeX=2 against a warm C=1
baseline, and the knee is reported as a floor, not a ceiling (EXPERIMENTS.md:120).

Acceptance

  • Both arms run; each arm's stub profile recorded and pinned via assert_stub_pinned, and the
    arm label verified against what /profile returns
  • Per rung: throughput, p50/p95, event-loop lag p99, RSS, per-turn worker CPU, sandbox-container
    CPU, lease saturation, over-admission events, spurious 429s
  • Duty basis row and connections per session recorded on every rung, not only sticky ones
  • The knee is attributed — worker CPU vs RSS vs replenishment vs admission cap — not just located
  • If the capacity arm still returns bound=not-observed, say so and name what bound first; do not
    report a floor as a ceiling
  • % of ideal throughput is not reported as a capacity figure when the ideal is stub-sleep-bound
  • Single-subject vs mixed-subject scope stated in the record, with the isolation claim made only
    if rossoctl/moca#239 landed
  • Results in deploy/vm/EXPERIMENTS.md, pointer added from docs/experiment-results.md (§5.6)

Open question to settle during the run

Is event-loop lag trustworthy as the capacity arm's headline diagnostic? It read ~11 ms flat at
every rung including c=1, where the tier is nearly idle — which looks like a sampling or timer
artefact. §5.2 makes it the metric that attributes a knee to worker CPU, so verify it against a
deliberately CPU-starved worker before relying on it. If it cannot be trusted, per-turn worker CPU
(#254 §2e) carries the attribution alone and the record should say so.

Related: rossoctl/moca-experiments#10 (spec + driver), PR rossoctl/moca#253 (routing prerequisite), rossoctl/moca#239 (mixed-subject scope), E9 run.

Lingua principale
Shell
Stelle
0
Fork
1
Merge medio
6h 16m
PR unite (30g)
7

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di rossoctl/moca-experiments

Tutte le issue di rossoctl/moca-experiments

Issue simili

Altre issue su Shell/Bash

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.