feat(sync): explicit inbox for mutations from unenrolled projects

Read the full issue and its 13-comment thread first — several decisions (lane choice, telemetry exclusion) were argued there. Start from the sync layer that writes sync_state rows per derived key and the doctor check sync_mutation_required_fields, plus the CLI surface around engram cloud enroll/cloud status. Done means: a reserved cloud:inbox target with list/claim/drop operations, a save-time warning (MCP routed_to field or CLI stderr), a per-key inbox_depth doctor check, and the closed-target-space invariant — all covered by tests.

Summary

Forward-looking design for what happens to sync mutations whose project is not enrolled. Today the target space is open: every cwd-derived key creates its own sync_state row, and mutations from unenrolled keys accumulate silently in the legacy cloud target. This proposal replaces that with a closed target space, a single reserved cloud:inbox, explicit human operations (list/claim/drop), and warnings that surface at save time.

The design was developed and reviewed against a real machine (Linux, engram 1.20.0, self-hosted cloud). All counts below were measured on that machine on 2026-09-05; project names and paths are anonymized.

Problem (measured)

1. A mutation whose derived key is not enrolled lands in the legacy cloud target with project = derived key. The worker only delivers enrolled projects → the row stays unacked forever.
2. The sync layer creates an empty sync_state row per derived key (cloud:<key>): lifecycle pending, no failures, no error of any kind — invisible by construction. (Measured on 1.20.0 — on 2.0.0-rc.6 the invisibility is gone: a new blocking doctor check fires on exactly this pathology. What it does and does not do: rc.6 note below the list.)
3. Measured result: 892 rows stuck + 35 phantom targets accumulating for ~3 months with zero alarms, because nothing failed — nothing was broken, nobody was looking.
4. Key derivation fragments. On one machine, within one backlog: one folder produced up to 3 keys (casing/separator variants); one non-repo working dir produced two different keys at different times (24 sessions under key A from July, 1 under key B in August — context-dependent resolution, not a typo variant); the basename fallback for non-repo cwds is exactly what generated the noise keys (/ → /, scratch dirs → their basenames, a UUID-named project dir, the user home dir → 538 sessions under its basename).
5. 880 of the 892 rows were session/prompt telemetry that does not go to cloud by policy — pure churn that should never have been enqueued.
6. And the per-project lane never delivers. Measured: the only lane the worker has ever delivered observations from is the legacy cloud target, grouped by the project column (a fresh observation for an enrolled project enqueued there was delivered in <60 s). The per-project cloud:<enrolled> targets hold 135 rows in their entire history — not one of them has ever been delivered. The lane this design uses as its central target model has a measured delivery record of zero.

Problem — rc.6 status of Problem 2 (measured 2026-09-06, same machine, 2.0.0-rc.6)

The 2.0.0-rc.6 doctor ships 7 checks (1.20 had 4). One of the new ones is blocking, and it fired live here:

[blocked] sync_mutation_required_fields
  non_enrolled_pending_mutations: Pending cloud sync mutations for project "scratchpad"
  are blocked because it is not enrolled.
  evidence: {"pending_mutations":2,"project":"scratchpad"}

The reproduction was accidental: a few commands run from a temp dir minted a phantom project key from the cwd — exactly this issue's pathology, reproduced without anyone trying to.

Honest status, without inflating or deflating:

• What rc.6 fixed: the invisibility. There is now a signal, and it is blocking — a phantom key can no longer accumulate for three months without a tripwire.
• What rc.6 does not do: route. The blocked mutations have no destination — no cloud:inbox (confirmed absent from sync_state), no claim/drop, no save-time warning. The detection has no exit.
• The check blocks on pure telemetry. The two live rows were one session and one prompt — zero observations. Per §1 that traffic does not go to cloud at all and should never be enqueued; rc.6 enqueues it and then raises a blocking verdict over it. (Both rows were later auto-acked locally without any push — the block bought nothing but an intervention.)

A blocking doctor verdict over two rows of cd noise is a worse remedy than the silence it replaces: 1.20 hid the problem, rc.6 halts on it. The design's core — a destination, human operations, and the save-time warning — is what closes the gap; the rc.6 check is a tripwire on the problem, not the routing this proposal asks for. I am reporting against the version currently under validation (2.0 RC), not the one already left behind.

Design

Central invariant

The target space is closed. Only these targets may exist:

• cloud:<project> for enrolled projects,
• cloud:inbox (reserved constant),
• cloud (legacy: transport lane for relations only, see §1).

A derived key never creates state of its own. The doctor enforces the invariant: any sync_state.target_key outside that set is an immediate finding.

Lane choice — explicit. This design proposes the per-project target as the delivery lane for observations: the worker reads cloud:<enrolled> and delivers from it; legacy stays frozen for new writes (the sole exception is the relations forwarding contract, §1). This is a change from the measured status quo (Problem 6: today only legacy delivers), and it is what makes item 1 real — routing enrolled mutations to a target nobody delivers from would be inventing a new way to strand rows. The controlled test (item 7) confirms that once a lane delivers, identical-content re-delivery works — so the lane change alone unblocks project moves; there is no dedup key to change.

1. Inbox mechanism

When an observation or relation mutation arrives whose key (resolved by the §4 pipeline) is not enrolled:

• target_key = cloud:inbox (constant, never derived).
• Column project = resolved key: slugged repo name, an unresolvable session designation (filed under the designated name, pipeline B.3), or unsorted.
• The payload's project field keeps the raw original (forensic, lossless).
• Telemetry (session, prompt) is not enqueued for cloud at all (local only). Codified policy: it does not go to the cloud. This eliminates the largest backlog source (880/892 of the stuck rows were telemetry).

Measured (2026-09-05): per-project targets cloud:<enrolled> contain only observations (135 rows in the whole history); all 4,799 sessions and 1,188 prompts live in legacy. So "telemetry does not enter per-project targets" is already de-facto behavior for enrolled projects; this design codifies and extends it (telemetry is not enqueued anywhere — today it goes to legacy and the worker auto-acks it: pure churn). Note the consequence for item 1: it is a real behavior change, not a formal one — today enrolled observations are enqueued to and delivered from legacy cloud (Problem 6); under this design, both enqueue and delivery move to cloud:<enrolled>. The failure mode is bounded: rows sitting undelivered in per-project targets — exactly the Problem 6 pathology, which the doctor's closed-target invariant surfaces instead of hides; and item 7's control shows that once the lane is live, identical-content re-delivery works, so no follow-up mechanism is needed.

How the inbox is kept distinct from real targets:

• The name inbox is a reserved constant: engram cloud enroll inbox is rejected; no project may be named inbox.
• The inbox sync_state row has its own lifecycle (inbox), not pending: no delivery cursor; the worker never "syncs" it; only tools list and count it.
• The doctor invariant protects it: any new non-enrolled target is a finding.

Legacy cloud is frozen for new writes, except relations forwarding. It changes from backlog dump to transport lane.

Relations — observed behavior (measured 2026-09-05, marked observed, not contract): the daemon re-enqueues only enrolled projects' relations toward the legacy target, and the worker delivers them there (167/167 historical relation acks live in legacy; 9 re-enqueued relations delivered in 12 s). The design formalizes that forwarding as contract (upstream item 6), it does not reimplement it.

2. Warnings

Design rule: a warning that depends on someone opening a log does not exist. This backlog lived for 3 months because nothing shouted.

Surfaces (a and b mandatory in v1):

a. At save time (primary, synchronous). The save response says it all (MCP: structured field routed_to: "inbox" + project_key; CLI: stderr line):

warning: saved to inbox: "<key>" (cwd <path>) is not enrolled — will not sync to cloud.
To enroll it: engram cloud enroll <project>

The only surface that cannot be ignored: it appears in the conversation where the memory was saved, at the moment the user decides whether that project deserves the cloud.

b. Doctor (audit, per-key tripwire with unsorted excluded). New inbox_depth check (warning), per key:

• For each distinct key other than unsorted: oldest row > 7 days → warning. A key waiting 7 days is a pending decision (enroll, claim, or drop).
• unsorted is excluded from the age tripwire: it receives noise by design forever, and an age warning would be permanent — alarm fatigue, reinventing the log nobody opens. Instead, a growth threshold: unsorted delta > 100 rows in 7 days → warning (tunable). Sustained noise does not alarm; a burst does.

c. Status (overview). engram cloud status always prints a line when the inbox is non-empty:

inbox: 214 memories in 9 unenrolled keys (oldest non-unsorted: 81 days)

(d. Optional, out of v1: session-start context line when the inbox grew since the last session.)

3. Lifecycle

The inbox does not drain itself, does not expire, has no TTL. Decisions about data are human.

Operations:

• engram cloud inbox list — counts + ages per key (read-only). For unsorted it breaks down by cwd (the payload keeps the raw cwd): the summary shows which directories compose the key.
• engram cloud inbox claim --project <key> [--as <enrolled>] [--cwd <path>] — moves those rows to the enrolled target (new INSERT + ack of the original; never delete — the mutation log is append-only forensics). --cwd is a selector inside unsorted: acts only on rows whose raw payload cwd is that path — without it, claiming unsorted would be all-or-nothing over unrelated directories. If --as names an enrolled project, the mapping is registered in the alias map (§4) and, for relations, the daemon forwarding does the rest (contract §1).
• engram cloud inbox drop --project <key> [--noise] [--cwd <path>] — acks as discarded, no delivery. --noise applies the noise pattern; --cwd restricts the operation to one cwd inside unsorted.

How the human finds out:

• The save-time warning (§2a) is the primary channel: every save routed to the inbox announces itself in its own conversation.

• The doctor per-key tripwire (§2b) is the backstop: if the warning escaped (another agent, batch flow), at 7 days the doctor shouts — for every real key, never for unsorted.

• The status line (§2c) gives the permanent overview.

• Enroll closes the loop: engram cloud enroll <project> inspects the inbox after enrolling; if rows exist for that key (or an alias of it), it prints the count and suggests the claim:

  $ engram cloud enroll <repo-name>
  ok: "<repo-name>" enrolled.
  -> 16 memories of "<repo-name>" waiting in the inbox.
     Claim: engram cloud inbox claim --project <repo-name> --as <repo-name>
  

Without this, enrolling <repo-name> tomorrow would leave 16 rows waiting in silence — the same design failure, at a smaller scale.

The only automation: post-enrollment relations forwarding (daemon, observed) and telemetry exclusion (never enters the queue). For observations and relations from unenrolled projects: zero automation.

4. Key derivation (two stages)

A v1 draft proposed key = slug(basename(cwd)) as the derivation rule, and review showed it was wrong.

What the data shows (sessions table, 2026-09-05):

• Current derivation is not basename: a git repo cwd resolves via git root (project_source: git_root). Basename is the fallback when git resolution fails.
• The backlog noise is exactly the fallback's output for non-repo directories: a non-repo model-staging dir, a non-repo scratch dir (prueba-like keys in saves while the session says another project), the user home (538 sessions), /, and a UUID-named app project dir.
• The same directory produces different keys at different times — a collision slugify cannot touch because it is not a typo variant, it is context-dependent resolution: one non-repo working dir mapped to key A in 24 sessions (July) and key B in 1 session (August).
• And there is a dominant path the v1 draft ignored: explicit project designation on the session. A non-repo agent workspace dir was designated as one enrolled project in 348 sessions. A pipeline starting at git root would send those 348 to unsorted and lose the attribution the agent gave.

Conclusion: promoting the fallback to the v1 rule institutionalizes the bug (the backlog is its output), and slugifying basenames would only rename the collisions. Slugify stays, but as a normalizer of the repo name, not as a derivator.

Derivation pipeline v1 (at enqueue time), in TWO stages:

The split is structural, not cosmetic: Stage A produces a candidate key; Stage B decides the target. Written as five sequential steps, an implementer codes five if ... returns — and then git root returns before the alias map is ever consulted, making key→project entries (renames) unreachable, which are exactly the repo case.

Stage A — resolve the candidate key (a single early return):

• Session designation present → candidate = the designated name; Stage A ends. No enrollment guard: if the designated project is neither enrolled nor in the alias map, the designation is kept anyway — Stage B resolves it (and if unresolvable, the inbox files it under that name, Stage B.3). Losing it here would betray the point of the step: it is the ideal inbox case — someone explicitly said which project this belongs to, it just needs enrolling. (The 348-session workspace case.)
• Else, git root: cwd inside a repo → candidate = slug(repo name). Slugify normalizes the repo name (lowercase, spaces/underscores → hyphen): a repo named LM_Studio_Model would yield lm-studio-model.
• Else, no candidate (no designation, no repo). Never basename: a cwd without a repo never creates its own key.

Stage B — resolve the target with the candidate:

1. Alias map, queried by candidate key and by cwd — two entry types:
   • key → project (renames): <old-folder-name> → <enrolled-project>. Reachable: git root no longer returns early, it only leaves a candidate.
   • cwd → project (non-repo working dirs): <working-dir> → <enrolled-project> — the direct fix for the measured collision.
     Hit → cloud:<enrolled-project>.
2. Match with enrolled: candidate == enrolled project → cloud:<candidate>.
3. Inbox: cloud:inbox with key = the candidate — an unenrolled repo keeps its identity (repo <repo-name> → inbox key <repo-name>), and an unresolvable designation is filed under the designated name — or unsorted if Stage A produced no candidate.

unsorted is the only key that can grow sustainably by design (hence its own growth tripwire in §2b, not age).

Out of scope (explicit)

• The cloud allowlist: untouched. Enrolling a new project still requires server-side authorization (self-hosted). The inbox does not bypass or extend it.
• What is worth syncing: claim/drop is human. No auto-enrollment, auto-drop, or memory expiry.
• The noise source: does not prevent agents from saving memories from scratch cwds (tmp, /, test dirs, home); it contains the damage (unsorted key, visible, growth tripwire) but not the saves. "Don't save from scratch" is a separate client-side policy.
• Historical designations are not rewritten: sessions already filed under a different key stay that way in the history; the alias/cwd map governs future derivation only.
• The relations delivery architecture: the design consumes it (legacy + forwarding) and asks for it to be formalized as contract upstream (item 6); it does not reimplement it.
• Semantic re-filing: the inbox key is the repo/cwd, not the topic. Re-filing by topic is a separate (semantic) problem, out of scope.
• Stage A without a tiebreaker relies on a data property, not a design guarantee: measured 2026-09-05, 0 sessions with more than one directory over their lifetime (group by id having count(distinct directory) > 1 = 0). The classic failure mode of session designation (designate a project, move to another repo, memories keep filing under the first) does not exist in this data. If long sessions crossing repos appear, the designation needs a tiebreaker: it wins while the cwd stays inside the designated project's root.
• Multi-node: assumes a single machine / single engram DB.

Upstream work items

1. Enqueue routing + delivery lane: enrolled → cloud:<p> (both enqueue and delivery — see the lane choice in the invariant and Problem 6); unenrolled → cloud:inbox (pipeline §4); telemetry → not enqueued.
2. cloud:inbox constant + its own lifecycle + rejection of enroll inbox.
3. Doctor: closed-target invariant + inbox_depth check per key (7-day age tripwire; unsorted on a growth threshold, age-exempt).
4. CLI: inbox list | claim | drop + enroll notification (pending-rows count for the key or its aliases + claim suggestion).
5. Two-stage derivation pipeline (§4): Stage A = session designation (single early return, no guards) → slugged git root → no candidate; Stage B = alias map (by key and cwd) → match with enrolled → inbox (unresolvable designation under the designated name; unsorted without candidate). Remove the basename fallback.
6. Relations forwarding contract (formalize the observed daemon behavior).
7. Correction to the export mechanism — controlled test (2026-09-06). The first draft of this item claimed the worker skips memories whose content matches an already-ACKED upsert (a "content-identity dedup"). A controlled test refutes that: a byte-identical re-enqueue of an already-ACKED observation (same target, entity, op, payload, and project column; only seq and timestamp differ) was delivered by the worker within one cycle on the legacy cloud lane. A content revision of an observation with ACK history was likewise delivered.
   Read with the 67-move and 135-straggler cases, the pattern is single-factor: every observed non-delivery sat on the per-project lane, which has never delivered (Problem 6); every delivery sat on the legacy lane. The worker reads only the legacy cloud target (grouped by the project column). There is no dedup key in the evidence; the earlier 7-case table (which confounded lane and change type in the move rows) is withdrawn as dedup evidence and stands only as evidence for Problem 6.
   Consequence for the design: project moves need no dedup-key fix. They fail solely because the per-project lane is dead. The fix is the lane itself (item 1 + the invariant), and the control shows that once a lane delivers, identical-content re-delivery works — so moves will flow once per-project targets become the delivery lane.

Local work (not upstream)

Executed or executable on the affected machine; separate from the upstream items above.

• One-time cleanup (done-ready, under backup): delete the 35 phantom sync_state rows (0 unacked, target ∉ {enrolled, inbox, legacy}) + one long-dead 403 phantom row. Empty state rows, not the log: the mutation log (sync_mutations) is untouched — it is append-only forensics, rows are retired via ack, never deleted or rewritten. Nothing else: the data debt is already drained (880 telemetry rows acked, 9 relations delivered) and 20 permanent seq gaps remain by append-only design.
• Stopgap canary (optional, reversible): a local job that compares sync_state.target_key against {enrolled, inbox, legacy} and warns on new targets — real-time drift detection, no code change. Stays in place until the doctor invariant (item 3) lands upstream.

Related

• #1007 — projects consolidate ignores piped stdin and proceeds with the full candidate set (the consolidation flow that item 1 (the lane) makes actually deliverable)
• #699 — unenrolled machines accumulate sync_mutations and the doctor's prescribed repair was circular (closed; this design is the forward-looking fix for cloud-configured machines)
• #578 — consolidate over-merges; cwd-based detection creates duplicate projects
• #570 — named remotes with per-project sync routing
• #104 — project aliasing system
• #136 — normalize project names to lowercase

What this issue is asking for

This is a design proposal from a long-running single-machine deployment (10 enrolled projects, ~7.6k mutation rows, self-hosted cloud) — every number in it was measured, not estimated. I am not asking for a PR; I am asking for a decision:

1. Confirm the measured lane behavior the design is built on: the worker reads only the legacy cloud lane (Problem 6), verified by the controlled test in item 7 (identical-content re-enqueue delivered on the legacy lane; every observed skip sat on the dead per-project lane). If that model is wrong, tell me where — the rest of the design holds or falls with it.
2. Decide the lane question in the invariant: per-project targets as the delivery lane (this design's proposal), or keep legacy as the one observations lane and document it.
3. If the direction is approved, I'd suggest item 1 (the lane) as the first slice, with the doctor invariant (item 3) as the guardrail, and the rest following in order.

I'm not going to open a PR before the direction is settled, but I can do something rarer than code review: validate any implementation against a real, messy, single-user deployment with a full history — the machine where this whole issue came from. Happy to run betas, hit the edge cases, and report back with data. If the first slice is approved, I can also prepare a PR implementing item 1 (the lane), with item 3 (doctor invariant) as the guardrail — say the word.

Risk / rollback

The routing change is client-side: if a version routes wrongly, the doctor detects it in the first cycle (closed-target invariant). The bounded regression is unenrolled mutations sitting in the inbox (ack-based, no deletes from the log) — not lost data. For the lane change (item 1), rollback is the inverse change.