Automatically rebuild stale indexes in the background after engine upgrades
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
- Issue type
- Feature
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- sqlite, typescript
- Domain
- backend-api-design, databases
Research direction
Start with EXTRACTION_VERSION and getIndexBuildInfo()/isIndexStale() in src/extraction/extraction-version.ts and src/index.ts, then read #710 and #940 for existing behavior and database-replacement constraints. Identify the project-open/query entry points and the session-transcript index’s versioning before scoping the rebuild lifecycle. Add regression tests for stale/current indexes, concurrent clients, failure recovery, source changes during rebuilding, and platform-specific handover; done means stale indexes refresh without blocking queries or exposing a partial replacement.
Written by the indexing model from the issue text.
Description
Problem
After an upgrade changes extraction or resolution output, an existing graph can remain on the old behavior even when every source file is unchanged. End users should not need to read release notes, understand which fixes affect stored index content, or choose between sync and index to benefit from the new engine.
Could CodeGraph automatically rebuild a stale index in the background when that project is next opened/queried, while continuing to serve the last completed index until its replacement is ready?
Existing foundation
Checked upstream main at 374b3b4209d93d8be5bc77ca487ba027e76e0758 on 2026-09-08. The latest published release listed by GitHub is v1.6.0 (2026-08-26). This is a feature proposal based on source inspection, not a claim of a newly reproduced release regression.
EXTRACTION_VERSIONalready distinguishes extracted-content changes from SQLite schema migrations. Its documented purpose is currently to drive a reindex recommendation/reminder.getIndexBuildInfo()/isIndexStale()already compare the stored extraction version with the running engine. Full indexing stamps that version; incremental sync deliberately does not advance it.- #710 introduced this detection and the reminder. #525 covers automatic updates when source files change; this proposal covers an engine change while source files remain unchanged. #1526 improves incremental/full-rebuild convergence after source edits, which is a related but different trigger.
- #940 provides related groundwork for reopening a replaced database in a running server on POSIX. Its implementation explicitly excludes Windows, so the handover needs platform-specific validation rather than assuming an open database can be replaced everywhere.
Proposed behavior
- Extraction/resolver changes that require existing content to be regenerated advance the extraction version, including fixes that change stored output. CLI/UI-only releases do not cause a rebuild.
- On the next normal project open/use, detect an older or missing extraction stamp and start a background rebuild into a separate replacement database. Continue answering from the last completed, readable index with an explicit stale/updating status. Users should not need a manual command or confirmation, and the expensive rebuild should not block ordinary queries.
- Concurrent clients share one rebuild. After it succeeds, catch up any intervening source changes and coordinate the switch to the replacement database and its fresh query caches. Publish the new stamp only after success. On failure, retain the old usable index and report the failure; never expose a partially built replacement as current. A short coordinated handover is acceptable, particularly where Windows file locking requires it. If no usable old index exists or its schema cannot be read safely, the nonblocking fallback does not apply.
- Apply the same compatibility principle to the separate session-transcript index, using its own reader/index version. Idle projects and temporary indexes are refreshed if and when they are used again; an upgrade does not need to discover and rebuild every database on the machine.
This would eliminate the need for downstream upgrade scripts to maintain a global database inventory and force full reindexes after every deployment.
Suggested regression proof
- Build an index with an older extraction stamp, leave source contents/hashes unchanged, and open it with the newer engine. Hold the background rebuild at a test barrier and prove queries still answer from the old index with an updating status. Release it and prove queries reflect the new extraction without reconnecting or running a manual command.
- Open the same stale project from two clients and assert that only one rebuild runs.
- Inject a rebuild failure and verify that the previous usable index survives and the compatibility stamp is not advanced; a later attempt can recover.
- Change a source file during the rebuild and verify the replacement includes the change at handover. Validate connection/cache handover on Windows as well as POSIX.
- Reopen a current index and verify that no full rebuild runs. Cover the corresponding unchanged-transcript case for the session index.
- Dominant language
- C
- Stars
- 73.2k
- Forks
- 4.7k
- Avg merge
- 2h 13m
- Merged PRs (30d)
- 418
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from colbymchenry/codegraph
-
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
colbymchenry/codegraph#2634 ·
Maintainers usually reply within 1 day
-
Windows: bundled Node v24.16.0 can abort at exit (nodejs/node#56645); please bundle Node >= 24.20.0Possibly taken @Ancienttwo claimed this 2 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 67/100
colbymchenry/codegraph#2617 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 Under an hour Newbie friendliness 72/100
colbymchenry/codegraph#2482 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
colbymchenry/codegraph#1900 · 2 comments · 5 reactions ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
colbymchenry/codegraph#1700 ·
Maintainers usually reply within 1 day
All issues in colbymchenry/codegraph
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
OpenPrinting/cups#1772 ·
Maintainers usually reply within 1 day
-
tech-debt
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
antoxa2584x/nfsu2-sw#44 · 1 reaction ·
Maintainers usually reply within 1 day