[ENV-B4] Prove Docker and rootless Podman workflow parity, isolation and adversarial recovery end-to-end

Start with tests/test_sandbox_parity.py and the accepted ADRs listed in the issue; then inspect the workflow surfaces and dependencies #271–#273. Build an auditable test plan around real Docker runs, deterministic fakes, concurrency isolation, crash recovery, and security regressions. Done means the acceptance criteria have reproducible evidence, including clear reporting when a live test is skipped.

Artifact: implementation-ready engineering issue. Parent EPIC: #266. Workstream: B. Size: XL. Baseline: main 711ff09 (2026-10-21). No implementation is authorized by this planning action.

Objective / scoped deliverable

Build real-backend Docker tests and deterministic fakes covering single REMOTE issue ANALYZE_EXECUTE, REVIEW, FIX and READY_FOR_MERGE; exact candidate/readback, validation and review certificate, controller owned effects and blocked handoff. Validate two parallel issue workers with L0/L1/L2, no cross-run container/worktree/object leakage, and crash-window replay. Include negative credential, network, resource and teardown tests; distinguish skip due to unavailable kernel/daemon from a pass. Document optional operator live smoke with own credentials on disposable repo.

Existing foundations and boundaries

Read EPIC #266 and relevant accepted ADRs 0002, 0004, 0005, 0006, 0008. The deterministic controller retains every workflow transition, Git/GitHub write, read-back, ReviewCertificate and merge authorization. Do not embed Sandcastle's run(), replace the Python agent providers or introduce an arbitrary privileged mount. Preserve the Linux bwrap baseline. Coordinate session/interaction overlaps with #147/#126. If state schema must change, obtain protocol-number coordination on #23 before the migration.

Dependencies

• Hard: #271 (ENV-B1)
• Hard: #272 (ENV-B2)
• Hard: #273 (ENV-B3)
• Hard: #290 (ENV-B0)
• Hard (outside this EPIC, added 2026-10-23): #281 (LAUNCH-E F2: step-outcome JSON escaping defeats redaction) and #284 (LAUNCH-E F5: a credential directly after an ANSI escape is missed). This gate asserts "no credential in logs", which is not valid until both land.
• This issue must not be considered complete by filing a follow-up for a core acceptance criterion; split decisions before implementation only if the EPIC ledger is updated.

Relevant implementation surfaces (starting points, not an exhaustive mandate)

tests/test_sandbox_parity.py, tests/test_sandbox_docker.py (new), tests/test_docker_workflow.py (new), .github/workflows/ci.yml, docs/usage.md

Acceptance criteria

• No mocked sandbox can substitute for real Docker E2E in the recorded supported-host evidence
• End-to-end exact candidate SHA and independent review certificate verified on review/fix loop
• Concurrent workers cannot affect each other's sandbox, candidate, logs, Git store or state
• Kill/restart matrix and unsupported-host skip reporting reproducible, plus explicit security regression evidence
• No container backend declared production-supported or EPIC MVP complete without real-run evidence
• Existing regression suite passes; focused negative and crash-window tests are attached to the PR, or a design-only issue includes an auditable test plan.
• Update documentation and backend capability/rollout status when user-facing behavior changes; do not represent a skipped live test as a pass.

Exclusions / anti-goals

Do not weaken sandbox.mode: required, allow agent-held GitHub write credentials, expose host .git common-dir/state to isolated agents, silently apply raw REMOTE diffs, or invent a second lifecycle controller. EPIC #255 Public Alpha is not blocked by this task.

Verification guidance

Use offline fake executable/provider fixtures first, then explicitly gated disposable-repository real-backend tests when applicable. Record exact candidate/base SHAs and externally observed failures, ensure secure redaction, and demonstrate idempotent cleanup and resume.

Provider acceptance clarification (2026-10-22)

• Test Claude Code and OpenCode 2.x key/owned-login routes and Pi API-key/OAuth routes per #290. Use fake credentials for CI, and only claim live subscription compatibility with opt-in verified evidence.
• Codex is explicitly NOT a release gate or MVP acceptance condition.

Required isolation/authority E2E matrix (2026-10-23)

• Hard: #292 (ENV-A5).
• Real Docker backend receives read-only exact-SHA host snapshot at /source; agent creates commits and compiles/tests only in private /workspace. Assert host input source and operator main checkout remain byte-identical before/after.
• Mock or stub container gh and attempt direct GitHub push/write with no credentials: must not authorize effects; inspect host Controller effect calls gh and verified host git only.
• Inject dummy model credentials via #290 and prove no values in image, docker run/exec argv, docker inspect, logs, Git bundle, build artifacts, GitHub comments or other worker's workspace.
• Assert exact commit identity on controlled import and full-review lifecycle, worker isolation, canceled build, crash/recovery and no host-side build execution.

Planning reconciliation (2026-10-23)

Artifact: planning update from the 2026-10-23 reconciliation of EPIC #266 with its architecture review (#266 comment). Baseline re-read: main 388ecb3. Planning only: nothing here implements a feature or checks off a criterion. The earlier sections stay authoritative where this one does not change them.

Native dependencies (verified read-back 2026-10-23): blocked_by #271, #272, #273, #290, #292, and #281 and #284 (outside this EPIC). This is the container MVP gate. It does not depend on #276, #277, #278 or #294, and #255 Public Alpha does not depend on it. #275 (rootless Podman) moved into the MVP on 2026-10-23 and is now a native prerequisite (blocked_by #275).

Revised coverage: the gate tests the reconciled contract

• Read-only exact-SHA /source snapshot and private writable /workspace. The host input and the operator's checkout are byte-identical before and after.
• Launch parity for Claude Code, OpenCode 2.x and Pi, with each route in #290's matrix. Codex is not a gate.
• The correct candidate SHA across ANALYZE_EXECUTE, REVIEW and FIX, read through #269's candidate-source abstraction. The ReviewCertificate binds that exact SHA.
• FIX is seeded from the current accepted (verified, imported) candidate, never from an earlier container's unverified workspace or from the original base.
• Validation and pre-merge (EXPORT_COMMAND) commands run only in an isolated container over the verified materialized tree, or are refused before work starts. No candidate command is spawned on the host.
• Host-only gh and hardened git effects. An attempted push or GitHub write from inside the container authorizes nothing.
• No GitHub token, gh configuration or credential helper, SSH agent, host HOME or Git common dir inside the container.
• Image digest and in-container provider version are verified and recorded. An unpinned or mismatched image is handled per #271 and #273.
• Runtime identity. Docker Engine and rootless Podman (a plain podman or the podman-docker wrapper) are each identified correctly; a Podman-backed docker is never treated as Docker Engine. Rootful runtimes and podman machine are refused.
• The real-backend E2E runs on both rootless Docker and rootless Podman on Linux (#275, moved into the MVP 2026-10-23). The evidence names each runtime. A runtime not available on the test host is reported as untested, never as passed.
• Source delivery follows #267 decision 12: a read-only bind only where the probe proved a shared filesystem, copy-in of the seed otherwise. Forced copy-in is tested on a local runtime.
• Backend pinning: a run created under bwrap refuses resume/step/unblock under Docker and vice versa, before anything is saved or launched. A run created under one container runtime refuses to continue when a different runtime is detected (#267 decision 8).
• Crash/restart cleanup at each #272 lifecycle boundary, and two concurrent workers that never touch each other's containers, volumes, caches, projections, candidates, logs or state.
• Bounded survival (#267 owner decision, 2026-10-23), on both runtimes:
  • after a controller SIGKILL mid-run, the container is gone within #272's measured bound without reconciliation;
  • with the watch disabled, it ends at the in-container deadline;
  • under required, a host that fails the death-watch proof is refused.
• Provider credential redaction with dummy secrets: no value in the image, docker run/exec argv, docker inspect, container logs, worker logs, step outcomes, Git artifacts, build output, GitHub comments, state.json, the lifecycle journal or configuration fingerprints. Valid only with #281 and #284 merged.
• Reuse is proven, not assumed: the snapshot comes from materialize() and the import from the generalized strict import of #269. A test fails if a second tree-export or import path exists.
• Explicit record of the network-egress limitation. The evidence and the docs state that a bridge network does not restrict traffic to model endpoints, and that residuals R3 and R4 apply (#270). Egress mediation is #294.
• No production-support claim from skipped tests. A skipped real-backend test is reported as untested, and the backend's rollout status says so.

Received from #292 (ownership transfer, 2026-10-23)

These are #292's criteria 1, 2 (second half), 3, 6 (real-workspace proof), 8 and 9 (E2E half), quoted verbatim on #292. They are proven here on real Docker. Criterion 8 in particular:

• Two concurrent worker containers cannot access each other's code, volumes, build caches, secret leases or artifacts; tests cover host HOME/GitHub token absolute paths and Docker socket denial.

Test organisation

• Real Docker tests are separate from the fake-backend unit tests: separate files and markers, opt-in, run only on a host whose capability probe passes.
• A hosted CI runner is not assumed capable just because it has a docker executable. The runner's capability is measured the way CI's sandbox job measures bwrap (make test-sandbox-measured). A runner that cannot enforce reports the real-backend tests as untested, and its result is not taken as a pass.
• The recorded supported-host evidence names the host, the engine (Docker Engine or Podman), rootless or rootful, and the image digest.