Automatically rebuild stale indexes in the background after engine upgrades

Start with EXTRACTION_VERSION and getIndexBuildInfo()/isIndexStale() in src/extraction/extraction-version.ts and src/index.ts, then read #710 and #940 for existing behavior and database-replacement constraints. Identify the project-open/query entry points and the session-transcript index’s versioning before scoping the rebuild lifecycle. Add regression tests for stale/current indexes, concurrent clients, failure recovery, source changes during rebuilding, and platform-specific handover; done means stale indexes refresh without blocking queries or exposing a partial replacement.

Problem

After an upgrade changes extraction or resolution output, an existing graph can remain on the old behavior even when every source file is unchanged. End users should not need to read release notes, understand which fixes affect stored index content, or choose between sync and index to benefit from the new engine.

Could CodeGraph automatically rebuild a stale index in the background when that project is next opened/queried, while continuing to serve the last completed index until its replacement is ready?

Existing foundation

Checked upstream main at 374b3b4209d93d8be5bc77ca487ba027e76e0758 on 2026-09-08. The latest published release listed by GitHub is v1.6.0 (2026-08-26). This is a feature proposal based on source inspection, not a claim of a newly reproduced release regression.

• EXTRACTION_VERSION already distinguishes extracted-content changes from SQLite schema migrations. Its documented purpose is currently to drive a reindex recommendation/reminder.
• getIndexBuildInfo() / isIndexStale() already compare the stored extraction version with the running engine. Full indexing stamps that version; incremental sync deliberately does not advance it.
• #710 introduced this detection and the reminder. #525 covers automatic updates when source files change; this proposal covers an engine change while source files remain unchanged. #1526 improves incremental/full-rebuild convergence after source edits, which is a related but different trigger.
• #940 provides related groundwork for reopening a replaced database in a running server on POSIX. Its implementation explicitly excludes Windows, so the handover needs platform-specific validation rather than assuming an open database can be replaced everywhere.

Proposed behavior

1. Extraction/resolver changes that require existing content to be regenerated advance the extraction version, including fixes that change stored output. CLI/UI-only releases do not cause a rebuild.
2. On the next normal project open/use, detect an older or missing extraction stamp and start a background rebuild into a separate replacement database. Continue answering from the last completed, readable index with an explicit stale/updating status. Users should not need a manual command or confirmation, and the expensive rebuild should not block ordinary queries.
3. Concurrent clients share one rebuild. After it succeeds, catch up any intervening source changes and coordinate the switch to the replacement database and its fresh query caches. Publish the new stamp only after success. On failure, retain the old usable index and report the failure; never expose a partially built replacement as current. A short coordinated handover is acceptable, particularly where Windows file locking requires it. If no usable old index exists or its schema cannot be read safely, the nonblocking fallback does not apply.
4. Apply the same compatibility principle to the separate session-transcript index, using its own reader/index version. Idle projects and temporary indexes are refreshed if and when they are used again; an upgrade does not need to discover and rebuild every database on the machine.

This would eliminate the need for downstream upgrade scripts to maintain a global database inventory and force full reindexes after every deployment.

Suggested regression proof

• Build an index with an older extraction stamp, leave source contents/hashes unchanged, and open it with the newer engine. Hold the background rebuild at a test barrier and prove queries still answer from the old index with an updating status. Release it and prove queries reflect the new extraction without reconnecting or running a manual command.
• Open the same stale project from two clients and assert that only one rebuild runs.
• Inject a rebuild failure and verify that the previous usable index survives and the compatibility stamp is not advanced; a later attempt can recover.
• Change a source file during the rebuild and verify the replacement includes the change at handover. Validate connection/cache handover on Windows as well as POSIX.
• Reopen a current index and verify that no full rebuild runs. Cover the corresponding unchanged-transcript case for the session index.