[Rewrite P1] Factory logistics: host split, worktree protocol, cost telemetry

Start with parent issue #860 and locate factory-spans.ts and the existing docs/rewrite/ runbook sections. Check how factory jobs, worktrees, and cost reporting currently operate before attempting the end-to-end proof. Done means both hosts have documented runbooks and a trivial job produces a Rust change built on tauceti, reviewed on chappie, with spans recorded and Actual Cost filled on the board.

Parent: #860
Milestone: Phase 1: Rust foundation
Host: both · Expected cost: $200

The host split and the working agreement the factory runs under.

• chappie (this host): factory orchestration, all review rounds, LMDB benchmark rig, differential harness driver, CI-first validation. Constraint measured: docker access is OPA-authz-scoped (labelled containers allowed; builds via BuildKit driver; no --load) — Rust builds here run in BuildKit via the container driver or natively on the host toolchain path the factory agents use
• Ada's host (flock id tauceti until 2026-10-20 09:00 ET, then Ada; container 68d12bce8009): release-grade builds and the 24-hour fuzz run. Consent recorded 2026-10-20T02:40Z, in Ada's words, below. Never hard-code the flock id — a peer addressing tauceti after the rename reaches nobody.
• Worktree protocol — what #914 fixed and what is still open is written below.
• Cost telemetry: factory spans script (factory-spans.ts) runs per job; the factory manager reads spans → Actual Cost USD per issue on the board; weekly variance roll-up as an issue comment on #860. Prices committed in 399b0905; they read $0.00 until the litellm restart.
• Model budget: a drafted policy is ready (private doc; not linked here because this repository is public). It needs two numbers from Cameri — a per-issue cap and a fleet weekly cap — before #865 opens. The draft's shape: one provider (the fleet gateway), a hard stop at the cap that escalates with the spans line rather than continuing, a weekly roll-up on #860, and heavy review rounds outside 13:00–19:00 UTC Mon–Fri.

Ada's runbook section (her words, measured on her host 2026-10-20T02:40Z)

What may run here. Build, test and soak of the nostream rewrite, from a worktree: cargo build --release, cargo test, cargo fuzz run <target>, and the soak binary. All of it under the scratch layout below.

What may not run here. Docker and compose: this host's authz policy denies docker exec and docker run, and the daemon you would be talking to also runs the live stack (35 containers, including the gateway and NATS). A compose project started here can disturb names and networks that carry production traffic. Also off-limits: writes outside the scratch tree; binding a port reachable from outside; ~/.flock, ~/.omp-agent, ~/.claude, ~/.ssh; anything under /workspace/containers or the live-stack checkouts except through git; any deploy, migration or restart command.

Resource caps. Four vCPU total, shared. Builds: taskset -c 0-2 (three cores), nice -n 10, and cargo -j 3. The soak: taskset -c 3, nice -n 19, ionice -c3. Memory: there is no cgroup cap available on this container — /sys/fs/cgroup is read-only and the init is herdr, not systemd, so systemd-run --scope will fail. The cap must therefore be a watchdog: stop the job if RSS passes 6 GB, and treat 6 GB as the ceiling in the worktree design too. Only 4 GB of swap exists, so a job that grows past RAM thrashes the box and stalls the session.

Disk. Budget 120 GB. 350 GB is free. target/ is the consumer: one target directory per phase, pruned when the phase ends, and no more than two phases' targets on disk at once. Say so if a phase needs more.

Scratch layout. Use /workspace/nostream-worktrees/phase-N/, not ~/nostream-worktrees/phase-N/. /home/agent is private to this container. /workspace is the host's bind mount, so it is the only path that can genuinely be shared. Targets at /workspace/nostream-target/phase-N/, logs at /workspace/.scratch/nostream/phase-N/.

First measured build on that host (2026-10-20T02:43Z, cold, rewrite/main at 6ef609a). taskset -c 0-2 nice -n 10 cargo build --release -j 3, empty target directory, exit 0:

The caps hold with room: the 6 GB watchdog is 4.5× the measured peak. Artifacts: release/nostreamd 4.1 MB, release/nostream-cli 2.5 MB. CARGO_HOME sits inside the toolchain directory, so the crates.io cache lands there too — .nostream-toolchain/ is 750 MB after one build (575 MB toolchain, 175 MB cache), and a phase that shares that CARGO_HOME starts warm.

Fuzzing is not ready to run, and the toolchain it needs cannot be installed from inside. The repository has no fuzz/ directory and no fuzz_target references, so there is nothing to cargo fuzz run yet — the targets are #868's work. On Ada's host, measured 2026-10-20T02:44Z: the session is uid 1001, sudo refuses before prompting (the one sudoers file is bind-mounted from the host with uid 1001 on it), and /var/lib/apt/lists/ is empty, so apt-get install -s clang reports no such package. apt cannot be used there without a host step.

Two ways to get clang, and neither is needed until #868 defines the targets:

1. Host step: add clang and lld to that container's image, or refresh the apt lists as root.
2. No root: unpack the prebuilt clang+llvm-19.1.7-aarch64-linux-gnu.tar.xz (HTTP 200; the 20.1.x asset names 404) under /workspace — 1,079 MB compressed, roughly 4 GB unpacked, inside the 120 GB budget. Nightly is the easy half: rustup installs it into the same RUSTUP_HOME without root.

Deferred deliberately: nothing fuzzes today, so a 4 GB unpacked toolchain would sit idle. Revisit when #868 opens.

Which jobs she takes. The release build and the 24-hour fuzz run. The 72-hour soak only at lowest priority, and it would measure code on a busy machine, because a deploy, a factory build or a console change preempts its cores for minutes at a time — so the 72-hour soak stays on chappie's side and tauceti takes a 24-hour one instead.

Prerequisites, measured. No Rust on that host: no cargo, rustc or rustup; cc, gcc, ld, make, pkg-config present, clang and cmake absent. Fuzzing needs nightly and clang. rustup-init is fetchable and git ls-remote [email protected]:cameri/nostream.git resolves rewrite/main.

Install into /workspace, not the container home — /home/agent is on the container layer and a recreate takes the toolchain with it (CARGO_HOME=/workspace/.nostream-toolchain/cargo, RUSTUP_HOME=/workspace/.nostream-toolchain/rustup, added to PATH in the job script).

Access path

Decided by measurement, not assumption: git only. Ada reported on 2026-10-20T02:42Z that /workspace/projects/.verify/share-probe.txt, written from chappie, does not exist on her side, and neither does the directory — her /workspace/projects holds onlygpus/ and skills/, nothing else. The two hosts mount different host directories at the same path, so no worktree is shared. She identified hers as a virtiofs mount, the Docker Desktop bind of a macOS directory.

The channel is therefore the git relay: chappie pushes rewrite/main and the phase branch; Ada fetches, runs, commits results to a branch, pushes; chappie merges. No new access, no open port, nothing out of band.

Worktree protocol

• scripts/rust.sh keys CARGO_TARGET_DIR to the checkout (/target-<basename>-<cksum>), so two trees can no longer read each other's artifacts — merged in #914, measured cost one cold build per checkout (36 s for cargo test --workspace --all-features --no-run).
• One worktree per phase; a phase's target directory is deleted when the phase ends, per the disk budget above.
• Factory jobs branch from the phase worktree; artifacts (benchmarks, fuzz corpora, soak logs) return as committed files under docs/rewrite/ or attached to the phase issue — never out of band.
• Because the filesystem is not shared (see Access path), a "shared worktree" is two checkouts of one branch. Ada's is /workspace/nostream-worktrees/phase-N on branch phase-N; chappie's is its own. Divergence is resolved by git, not by copying files: she commits under docs/rewrite/ and pushes, chappie merges.

Acceptance: both runbook sections written (chappie's is inline above, Ada's is quoted verbatim); the access path decided by measurement (git relay, see above); one end-to-end proof — a trivial factory job (spec → build → review → approve) lands a Rust change built on Ada's host, reviewed on chappie, with spans recorded and the board item's Actual Cost filled.