Make lease requests durable across restart

Start with src/core/domain.ts, src/core/registry.ts, and src/core/lease-acquisition-coordinator.ts, then trace startup recovery in src/core/startup-converger.ts. Use the listed behavioral tests as the acceptance map, especially replay, restart, concurrency, pruning, and legacy state loading. Done means request state is core-owned and durable, LeaseRequestTracker keeps only streaming and long-poll plumbing, and the specified contract and documentation updates are complete.

Part of #70. See ADR 0021.

Purpose

Store each lease request before Simlock queues or provisions work.

A client must recover the same result after a disconnect or process restart. A retry must not create a second lease.

Simlock must keep this state in the core registry. A frontend must not own a second request registry.

Scope

Today a lease request exists only as a promise in memory. Nothing on disk names it, so a client that loses its connection cannot ask what happened, and a daemon restart drops every queued request without telling anyone. The HTTP frontend works around this with its own in-memory registry (LeaseRequestTracker), which socket clients cannot see and which dies with the process.

After this task the registry stores a lease request the way it already stores devices and leases. Repeating a request returns its stored result instead of an error. A restart leaves every request in a state a client can read. The HTTP tracker keeps its streaming and long-poll plumbing and stops keeping its own copy of the truth.

Required behavior

• Store the request before queue admission and before any driver work.
• Key a request on (requesterId, idempotencyKey).
• Accept idempotencyKey as optional. Store a request that carries none; it simply cannot be replayed.
• Return the stored result when a client repeats a request under the same key.
• Attach a repeat of a request that is still open to the existing wait. Do not start a second one.
• Return IDEMPOTENCY_CONFLICT when the same key arrives with a different device spec.
• Store ownerId on the record. Refuse a replay from a different principal with FORBIDDEN.
• Treat a capacity, disk, validation, or provisioning failure as terminal for that request.
• Never re-evaluate a terminal result under the same key, whatever changed on the host since.
• Admit a new key as a new request once the prior one is terminal.
• Keep REQUESTER_ALREADY_LEASED as the answer when the prior request is still open under a different key.
• Treat disconnect, request timeout, and abort as the end of that caller's wait only. None of them writes a result.
• Write a cancellation only through an explicit cancel.
• Settle every open request as terminal failed at startup. Say that the daemon restarted.
• Keep a terminal record for a configured window, then prune it.
• Cap the number of stored records.
• Serve the same record to every frontend on the daemon.

Recovery is a repeat of the request, not a new operation. A client that lost its response sends the same request again and gets the same answer.

Technical spec

Modules touched

• src/core/domain.ts — LeaseRequestRecord, its states, and the one predicate that answers whether a record is terminal.
• src/core/registry.ts — the third record type beside devices and leases: create, settle, read by key, prune, and their #commit/#restore halves. A state file written before this task loads without a request list rather than failing to start.
• src/core/lease-acquisition-coordinator.ts — write the record inside decisions.run before admission; replay a stored result; attach a repeat to a live waiter; settle the record wherever the request settles today.
• src/core/startup-converger.ts — settle every non-terminal record as failed before admission opens.
• src/core/config.ts — lease.requestRetentionMs and lease.maxRequestRecords, their defaults and validators.
• src/contract/schemas.ts — the config block and the request record's own schema.
• src/contract/operations.ts — idempotencyKey on lease.request.
• src/contract/errors.ts — IDEMPOTENCY_CONFLICT.
• src/daemon/dispatcher.ts — thread the key through lease.request.
• src/http/tracker.ts — delete #requests, #idempotency, #leaseRequestId and their timers; read core state; keep SSE and long-poll.
• src/simlock-client/client.ts, src/simlock-client/types.ts — the new input field.
• docs/CONFIGURATION.md, docs/HTTP-API.md, docs/EVENTS.md, docs/internal/EVENTS.md, docs/internal/KNOWN-PITFALLS.md.

The record must be written inside the same serialized section that checks uniqueness. Two concurrent requests under one key both pass the check otherwise. This puts a state.json write inside SerializedDecision.run, against that class's own note about long I/O. Accept it: leasing a device costs seconds to minutes, and a file write does not register next to booting a simulator. Do not move it out without a measurement.

idempotencyKey is caller-supplied and so is requesterId, so the pair is entirely under the caller's control. The record's ownerId is what authorizes a replay — the same guard lease.cancel already applies through pendingRequestOwner, for the same reason.

Contract and event changes

• lease.request gains an optional idempotencyKey. Additive; every existing caller keeps working.
• IDEMPOTENCY_CONFLICT — new error code, kind: "domain", HTTP 409.
• lease.rejected — reason gains daemon-restarted, emitted once per request settled by startup recovery. This widens a published vocabulary: record it in both EVENTS.md cuts, and say there that a consumer must tolerate a reason it does not know.
• lease.requested — payload gains the request id, so an observer can correlate an event with a stored record.
• No new operation. Replaying lease.request is how a client reads its request back.

Rules in play

• architecture.md 8 — the daemon owns this state. A frontend renders it and stores nothing.
• architecture.md 10 — one request per requester is already enforced in LeaseAcquisitionCoordinator. Replay reads that rule, it does not restate it.
• architecture.md 12 — startup settlement exists so a request never survives in a state nobody drives.
• safety.md — requesterId and idempotencyKey are claims. ownerId is what the daemon proved.
• events.md 6, 8 — the lease.rejected widening and the lease.requested payload change are documented in this change.
• testing.md — every test title below is a claim its body must prove.

Tests

• A repeated request under the same key returns the first result and grants no second lease.
• A repeat of an open request attaches to the existing wait instead of starting a second one.
• A request is on disk before the wait queue has seen it.
• A daemon restart settles an open request as failed and names the restart.
• A terminal capacity failure stays terminal under the same key after capacity frees up.
• A new key after a terminal result starts a fresh request.
• The same key with a different device spec is IDEMPOTENCY_CONFLICT.
• A replay from a different principal is FORBIDDEN.
• Two concurrent requests under one key produce one record.
• A disconnect ends the caller's wait and writes no result.
• A request carrying no idempotency key is still stored and still settled by restart recovery.
• A terminal record is pruned once the retention window passes.
• The record cap evicts the oldest terminal record and never an open one.
• A state file written before this task loads.
• GET /v1/lease-requests/{id} answers from the core record after a restart.

Done when

• LeaseRequestTracker holds no request state of its own.
• A socket client that reconnects after a disconnect recovers its result by repeating the request.
• docs/HTTP-API.md's lifecycle section no longer documents the 404 → re-request → 409 → GET loop, which only existed because the tracker died on restart.
• docs/internal/KNOWN-PITFALLS.md's "The HTTP tracker and notice buffer are the last frontend-held state" entry is split: the tracker half is resolved, the notice buffer half stands on its own.
• docs/internal/KNOWN-PITFALLS.md gains an entry for the gateway gap: a fleet request is durable only once it is routed to a worker, because a gateway persists nothing but its drained-worker list (ADR 0005, Decision 3). Extending durability to the gateway needs its own ADR.
• Both EVENTS.md cuts carry the lease.rejected and lease.requested changes.

Out of scope

• Supersession. No use case has been named for swapping a queued request for a different one, and it cannot swap a request whose device work is already in flight — the case cancelPending already refuses. A requester that changes its mind cancels and asks again; both calls are durable under this task.
• The external fence and admin-confirmed activation. It protects an identity that #74 mints and whose lifetime #75 governs, so it cannot be built before either.
• Managed identity status and removal acknowledgement. Same dependency. #76 needs this for warm identities that never receive a lease; it should be specified with #76 in view, not here.
• LeaseNoticeBuffer. Keyed by lease, not request, and it solves absence between polls rather than replay of a lost response. Its one irreducible fact is why a lease ended, which nothing durable records today.
• Durable requests behind a gateway. See Done when.

Depends on

• #71

Approval

• Approved for delivery

Written by an agent.