[DEL-A1] ADR: Controller Authority, Native Memory, Deterministic Task Contracts, and Crash Recovery

Read foundations in issues #158/#176 and ADR 0004 to understand Controller authority. The deliverable is a dated ADR (likely in docs/adr/) specifying state machine, memory layers and fallback protocol. Cover acceptance criteria with deterministic fake-engine scenarios and preserve single-agent regression tests. Done when all checkboxes met and schema coordinated on #23.

Parent: EPIC #315 · Stage M0 (architecture and baseline) · Size L · Reconciled 2026-10-25 (UTC) against main at f04db5f (state protocol 12). "Current scope" is authoritative. The earlier planning text follows verbatim and still applies where it does not conflict.

Current scope (reconciled 2026-10-25)

Ratify one dated ADR, under the next free number in docs/adr/, that defines three separate concerns: authoritative task state, the memory architecture, and the capability and scope decisions. Use the smallest contract that lets the Memory MVP (#317, #333) ship first and the multi-agent MVP (#332, #319) after it. No new orchestration daemon, no new privileged agent runtime, no new GitHub effect kind.

1. Authoritative task state

Define, as controller state below the existing phase:

• Identities: TaskID, AttemptID, RunID (the existing run key), role, candidate identity (base and candidate SHA), phase, policy version, provider profile.
• Lifecycle and legal transitions. Candidate states to evaluate: PROPOSED, ADMITTED, RUNNING, VERIFYING, ACCEPTED, RETRYABLE, BLOCKED, FAILED, CANCELED. They are design candidates, not permission to duplicate workflow states: the workflow's phases and its BLOCKED stay as they are, and a task state that mirrors a workflow state says how the two relate. Define the exact transition table, and which transitions are persisted in existing controller state and which are derived.
• TaskResult versus PhaseResult. A task result is a bounded, typed claim that the controller verifies. Only the existing phase acceptance path produces a phase transition from a CONTROL_RESULT. Today _invoke_phase parses every successful launch as the phase result (src/autoforge/engine.py:13111).
• Decision inputs and outputs of the reducer (#332). In: recorded task state, policy snapshot, provider capability observations, verification outcomes, budgets. Out: typed action proposals.
• Launch intent: what is persisted before a launch, so that a crash on either side of it is reconciled without a second launch or a lost result.
• Failure states, crash and replay behaviour, and resource-consumption records (physical launches, runtime, storage, attempts).

2. Memory architecture

Ratify four layers and the trust boundary between them:

1. Execution evidence: existing, authoritative, controller-observed (ADR 0004, ADR 0006).
2. Issue Working Memory: controller-owned, typed, scoped to repository, issue, run, phase and candidate (#317).
3. Repository Knowledge: optional, approved, repository-scoped, advisory (#325, in EPIC #331).
4. Retrieval projections: replaceable, rebuildable, never an authority (#324, in EPIC #331).

Then:

• Define the trust boundary between a model-proposed finding and a controller-observed fact.
• Define the commit point, and the reconciliation order after a crash, between the authoritative controller checkpoint, the Memory revision, the candidate SHA, the attempt result and the budget consumption. Say which one wins after a crash between any two of them.
• A Memory record is never a substitute for task state.
• Storage: decide the authoritative storage under SafeRoot's guarantees (descriptor-relative addressing and atomic publication; src/autoforge/safefs.py:590, state.py:2154). Do not assume SQLite meets them because FTS5 is the planned search backend. A pathname-opened database and its journal or WAL sidecars do not inherit them by default. Authoritative storage and a rebuildable search projection are separate decisions.

3. Capability and scope decisions

Record a decision for each:

• First supported scope. REMOTE ANALYZE_EXECUTE is the proposed smallest MVP. FIX and LOCAL follow later, or the ADR says why they ship together.
• Provider fresh-session capability. How each allowed configuration proves a fresh context (today Claude --no-session-persistence, OpenCode run --standalone, Pi --no-session), and which profile options or extra arguments void that proof.
• Interaction and pause ownership. #126 owns durable pause/resume. Until it lands, an interaction request or a budget exhaustion uses the existing safe stop (pause_available=False, engine.py:12930).
• Crash windows introduced by task launches and Memory writes.
• Storage security, including the same-UID residual under sandbox.mode: disabled, which is logical isolation only.
• Memory projection freshness: what a projection states about its revision, staleness and omissions.
• Protocol migration. Each persisted change is listed for #23, which assigns numbers. This ADR claims none.
• Unsupported provider behaviour: an explicit refusal before any side effect.

Dependencies

• Hard: none.
• Blocks (native): #317, #332, #333, #319, #321; and #324, #326 in EPIC #331.
• Coordination: #23 (protocol), #147 / #154 / #126 (provider capabilities, usage, interaction), #176 (locks), #266 / #269 / #278 (sandbox and candidate lifecycle).

Acceptance criteria (added 2026-10-25)

• The ADR defines the task-state identities, the lifecycle, the legal transitions, the TaskResult/PhaseResult split and the launch-intent protocol, with a transition table and a crash-window table.
• It ratifies the reducer's input and output types for #332, or names exactly what #332 decides.
• It ratifies the four memory layers, the trust boundary, and the commit order of checkpoint, Memory revision, candidate, attempt and budget.
• It records every capability and scope decision listed above, including the first supported phase and mode.
• The Memory MVP is implementable without #332, #319, #324–#326, SDK-only providers, Docker or Podman, or an embedding service.
• Its persisted-state needs are listed on #23; no number is assigned.

Earlier planning text (verbatim)

Superseded where it conflicts with "Current scope" above. In particular:

• The initial multi-agent MVP is deterministic (#332, #319). An LLM Manager is an optional adaptive mode, not the default coordinator.
• #324–#326 now belong to EPIC #331.
• The ADR's memory decisions are scoped so the Memory MVP needs none of #324–#326.

Parent: EPIC #315 · Wave A · Size L · Planning baseline f4fd5ef (2026-10-25 UTC). Research: GVS5H, not an implementation dependency.

Objective / deliverable

Ratify a precise manager/worker/verifier state machine inside a single ControllerEngine phase; roles are no new authority. Specify task/attempt/candidate IDs, single mutating worker, current candidate binding, resume/crash windows, stop conditions, legacy single-agent parity, interaction handoff and provider/session capability fallbacks. Define Ledger trust, source provenance, storage and state protocol impact. Decide whether Phase ANALYZE_EXECUTE and FIX get the same coordinator; review never becomes self-approved.

Dependencies

• Hard: none (can start from accepted #158/#176 foundations).
• Coordination only: #147 SDK providers / #126 interactions, #154 usage, #176 scheduler and #266 sandbox. Do not add blanket cross-EPIC hard blockers without evidence.

Acceptance criteria

• Dated ADR explicitly maps all transitions to existing Controller authority and validation/review/merge gates
• No GitHub write credentials or generic tool/shell escape for Manager, Worker, or verifier
• Deterministic fake-engine scenario matrix covers timeout, crash, canceled work, data tampering and candidate drift
• Migration/protocol needs coordinated on #23 before implementation; no number assigned
• No new EPIC scheduler; #176 lock domains remain authoritative
• Regression tests preserve original single-agent path, ADR 0004 controller-owned effects, exact candidate validation, ReviewCertificate and merge safety; provider credentials remain isolated.
• Docs/doctor/config compatibility updated where behavior changes; skipped real-runtime evidence never reported as pass.

Invariants / exclusions

The LLM Manager is advisory. Controller exclusively owns done acceptance, typed GitHub effects, state transitions, candidate identities and merge. Inner tasks are not GitHub native sub-issues and do not modify EPIC graph. Neither Agent nor verifier may execute repository-controlled commands outside the existing sandbox. No unbounded raw transcript/reasoning recording or ambient credentials. Feature remains opt-in; no Public Alpha #255 gate. Coordinate persisted schema on #23 before migration. Do not copy GVS5H's arbitrary subprocess verifier or its text-hash scratchpad and call it production recovery.

Verification hints

Use deterministic fake provider plus real existing CLI integration where available; include redacted bounded evidence, attack cases, failure/cancel/resume, explicit run/issue/candidate SHA. No new orchestration daemon or scheduler.

Native Memory architecture decision — TenseMem / Beacon / LTSearch synthesis (2026-10-24)

Decision to capture in the dated ADR: AutoForge Native Memory Engine + Controller-owned MemoryStore + SQLite FTS5 default retrieval + optional LTSearch advanced retrieval. Memory state/authority and Memory search projection are separate contracts. TenseMem inspires corrections/freshness/deterministic facts; Beacon inspires review-gated cross-issue experience promotion; GVS5H inspires curated in-issue handoffs. No direct mandatory runtime dependency on any of the three.

• ADR distinguishes Layer 1 authoritative existing Execution Evidence (verified Git/Review/CI receipts), Layer 2 Issue Working Memory (typed, ephemeral/replaceable, repo/issue/run/phase/candidate scoped), and Layer 3 Repository Knowledge (approved, durable, repo-scoped, advisory). Layer 2/3 must never become Controller state, GitHub write authority, permissions, accepted ReviewCertificate or merge evidence.
• Define canonical MemoryStore ownership, typed evidence-based state transitions, first-class correct/retract/supersede, applicability ranges, provenance, approval, rejection persistence, retention, projection version and freshness watermarks. LLM proposes; deterministic code decides and applies. Stale memories fail closed or visibly degrade; corrections cannot resurrect on replay.
• Define a separate replaceable MemoryRetriever read/index API and freshness contract. Default: local SQLite FTS5, no daemon/network/embedding requirements. #324 independently evaluates LTSearch (Rust, Tantivy BM25, LanceDB vectors, RRF) as optional adapter only, with code license/redistribution verification, memory budget, platform compatibility and independent measured go/no-go. Never equate search rank with fact authority or allow the search index to authorize record visibility.
• Define #325 review-gated per-repository Knowledge Promotion, optional read-only context projection for #319, and zero default cross-repository sharing. Approval is explicit, provenance-backed; memory suggestions or evaluator scores cannot self-authorize.
• Explicit trust barrier: Controller owns memory and retrieval (host-side); Claude Code/OpenCode 2.x/Pi only receive bounded, sanitized read-only context with scope/SHA/freshness labels. Do not install Beacon plugins/skills/telemetry collectors inside isolated providers or relax safe-mode / Pi --no-extensions --no-skills. No LLM token, raw transcript, GitHub credential or full diff entered into long-term knowledge.
• Resolve schema/write ordering/migration, index lag, read-after-write and immediate visibility of retractions; coordinate any persisted protocol number with #23 before assigning one. The existing single-agent path remains unchanged and feature disabled by default.

Research sources: TenseMem architecture, Beacon reviewed project memory, LTSearch query architecture.

MemCurrent architecture amendment — online/offline memory and bounded recall (2026-10-25)

Reference: MemCurrent design. The ADR MUST distinguish online Issue working-memory correction, offline derived Repository Knowledge proposals, and read-only recall, without assuming raw episodic transcripts or hosted Databricks services.

• Specify that Layer 1 verified execution evidence is canonical, Layer 2 #317 issue-specific notes/facts are promptly corrected, and Layer 3 #325 approved derived knowledge is durable but advisory. #326 offline consolidation is a proposal generator, not an authority granting automatic trust.
• Define Controller-generated compact, immutable-per-launch Memory Catalog / Index (scope, current revision, topic counts, freshness, citations) and optional search_memory → fetch_memory calls under #324/#319. Never inject entire corpus; searches/fetches are bounded, explicitly scoped and revalidated against MemoryStore.
• Define input watermark, incremental export, idempotent checkpoint/partition ownership and replay boundaries between #317, #325 and #326; a revoked lesson becomes unavailable immediately even before index or consolidation catches up.
• Explain why current GitHub/CI state (e.g. whether PR open) is not a durable remembered fact when Host Controller can query it; preserve only evidence-backed stable engineering lessons with applicability and revalidation rules.
• Define what opt-in/manual or scheduled consolidation may do, provider/sandbox rights and budget/consent; never ingest unbounded transcripts, auto-install Agent Skills, change Controller effects or ship as part of Public Alpha.
• Evaluate and document the tradeoffs of online-only, offline-only and hybrid write paths on AutoForge outcomes rather than treating Databricks internal benchmark gains as transferable.