Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

bug(sync): autosync blocks local writes every cycle while a non-enrolled legacy backlog exists (v3.6.0)

Open
  • Is something wrong?
#2,166 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
45/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
go, sqlite
Domain
backend, database

Research direction

Read internal/cloud/autosync and internal/store/store.go, especially Manager.push and QuarantineIrreparableSyncMutations. Trace why each poll scans pending cloud mutations for projects that are not enrolled and how that holds the SQLite write lock; the report provides a reproducible large-backlog scenario and an offline server setup. Done when autosync no longer repeatedly blocks local requests on undeliverable backlog rows, while enrolled project sync continues to work.

Written by the indexing model from the issue text.

Description

status:needs-review type:bug
Existing Issue Search
  • I searched existing issues and will add evidence to a canonical tracker when applicable.
📝 Observed Consequence

After enabling cloud autosync (one enrolled project), the local engram serve stops responding for most of every 30-second cycle:

  • Agent memory writes time out (mem_save, mem_delete, session registration). One delete was applied but reported as unconfirmed.
  • GET /health and GET /sync/status time out (2 s client limit) for about 25 of every 30 seconds.
  • engram doctor fails with migration: database is locked (5) (SQLITE_BUSY).
  • The process uses close to one CPU core continuously.

The enrolled project itself syncs correctly.

The store holds 35,239 pending cloud mutations from 85 non-enrolled projects, journaled from 2026-05-20 to 2026-09-13, before cloud was configured on this machine. None are newer.

🔄 Steps to Reproduce
  1. Take a store with a large backlog of pending cloud mutations for non-enrolled projects (here: 35,239 rows, ~25 MB of payload, 228 MB database).
  2. Run engram serve with ENGRAM_CLOUD_AUTOSYNC=1, a cloud server and a token.
  3. Poll GET /health every few seconds across the first poll cycles.

No network is needed: I reproduced it on a copy of the store with ENGRAM_CLOUD_SERVER=https://127.0.0.1:1 (unreachable).

✅ Expected Behavior

A backlog that autosync never delivers should not block local writes on every cycle.

❌ Actual Behavior

On the store copy with the unreachable server, /health timed out (>2 s) when the first poll cycle ran, at about 40 s. The process had used 14.3 s of CPU by then.

The same copy with only those 35,239 rows changed behaves normally. Same measurement (engram 3.6.0, offline):

Store copy /health at the first cycle CPU after ~40 s Autosync phase after the cycle
Unchanged timeout (>2 s) 14.3 s no answer
Rows moved to target_key='cloud:inbox' ~0.1 s 1.9 s degraded / transport_failed (expected offline)
Rows set to disposition='superseded' ~0.1 s 2.2 s degraded / transport_failed (expected offline)

Both changed copies kept every memory table and the sync_mutations row count identical; nothing was deleted. Running engram cloud enroll <project> on them backfilled the project's current state (67 observations, 23 sessions, 47 prompts, 12 relations), which matches its live rows.

Workaround applied on the live store: the inbox variant, in one transaction. Afterwards /health answered all 30 probes over ~65 s, the process used 2.4 s of CPU, autosync reported healthy, and a new memory of the enrolled project saved at once and reached the cloud.

Operating System

Windows

Release Version (Report Context)

3.6.0

Agent / Client

Other

📋 Relevant Logs
[autosync] started (server=https://<redacted>)
[autosync] quarantined 214 irreparable pending mutation(s); inspect with `engram doctor --check sync_mutation_required_fields`
engram: engram: migration: database is locked (5) (SQLITE_BUSY)

/sync/status last_error: "pending cloud sync mutations are blocked because project(s) are not enrolled: … (85 projects)"

💡 Additional Context

Client: Pi with gentle-engram, which starts engram serve with ENGRAM_CLOUD_AUTOSYNC=1. OS: Windows 11 Pro.

Possible cause. This is my reading of the source, not proven by a test:

  • Manager.push calls QuarantineIrreparableSyncMutations(targetKey, "", true) on every cycle: the 30 s poll, plus the debounce after each write.
  • With an empty project, that scans every pending cloud row of every project, including non-enrolled ones. It validates each row inside withTx, and the store opens transactions with _txlock=immediate, so the write lock is held for the whole scan.
  • The backlog is never delivered, so the scan repeats every cycle.
  • internal/cloud/autosync and internal/store/store.go are unchanged between v3.6.0 and main @ 9824739.

Related:

  • #1896: same symptom, different path (enrolled repair backfill); closed.
  • #1013: non-enrolled backlog; list/claim/drop not shipped.
  • #1188: inbox. docs/CLOUD-INBOX-STATE.md says legacy migration is not implemented, so moving old rows to the inbox has to be done by hand with SQL.
Dominant language
Go
Stars
7.1k
Forks
729
Avg merge
9h 12m
Merged PRs (30d)
331

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Gentleman-Programming/engram

All issues in Gentleman-Programming/engram

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.