Managed shared mode on a large Windows repo: 25–32 min cold readiness, watcher input churn, scheduler stall, query cliffs (v1.2.3)

Start with the referenced paths: tgrep-core/src/generations/mod.rs, tgrep-core/src/generations/git.rs, tgrep-cli/src/serve/shared/managed.rs, tgrep-core/src/managed/views.rs, and tgrep-cli/src/serve/shared/protocol_v2.rs. Read the issue’s measurements and suggestions, then trace the relevant build, watcher, scheduler, and query paths. Done means addressing and verifying the reported readiness, invalidation, scheduler, response-cap, and JSON fidelity problems; the issue may need to be split into smaller tasks.

Summary

We embed tgrep serve --shared (managed-v2, v1.2.3) as a shared daemon for several linked worktrees of one repository, and qualified it against ripgrep on a large Windows monorepo (~293k files, 6.9 GB). Every query served by a managed generation matched ripgrep exactly. The problems are time to readiness and runtime behaviour under real conditions. They are grouped below with evidence and code pointers; happy to split into separate issues.

Environment

• Windows 11, ReFS Dev Drive (reported as a developer volume but not trusted), Defender real-time protection on, plus an endpoint management/security agent that periodically walks the tree.
• tgrep 1.2.3 release binary, notify 7.0.0, 16 verification workers.
• Host policy: work.workers = 1, 8 GiB private work and staging, 16 GiB repository storage.
• Seven campaign runs. Each run had one cold worktree, two linked worktrees at the same commit, and one session that reused an existing view.

1. Cold readiness takes 25–32 minutes

Readiness timings in minutes. "Build" is base_materialization_us. "Queue" is time spent waiting behind another worktree's operation.

Run	Cold total	Cold build	Linked worktree 1	Linked worktree 2
c	31.4	25.4	14.1	14.5 + queue 14.2
d	24.9	19.2	6.1	6.1 + queue 6.2
e	25.9	20.1	6.5 + queue 2.3	6.1 + queue 8.8
f	25.4	19.6	6.1 + queue 6.4	6.4
h	25.7	19.6	6.5	6.6 + queue 6.5
i	25.6	19.7	6.3 + queue 6.2	6.2
j	31.8	18.8	6.3	6.2 + queue 6.3

A session that reused an existing view was ready in about 2 s.

Where the time goes:

1. Base generation build: 19–25 min.
   • GenerationManager::build_into (tgrep-core/src/generations/mod.rs:964) is a serial loop over unique blobs. It is fed by a single git cat-file --batch process (tgrep-core/src/generations/git.rs:387, :471).
   • Each blob is read, decoded, binary-checked, given a ContentId, and has its trigrams extracted into the external sorter, all on that one path.
   • That is about 287k blobs (6.9 GB) at about 6 MB/s, producing a 3.15 GB generation.
   • Only the first worktree at a commit pays this cost. Linked worktrees reused the generation in about 0.6 s.
2. Full first-attachment verification: about 6 min per worktree.
   • The first-attachment-or-restore full-repair trigger (tgrep-cli/src/serve/shared/managed.rs:353) opens and reads all ~293k files, even when the worktree is clean at the base commit.
   • On the cold worktree this added 6–13 min after the build.
   • Summed worker_open_us was 1,100–1,240 s across 16 workers, about 3.9 ms per open. Filesystem filter overhead is the likely cause, but that is a correlation only.
3. Queueing. With one work slot, a second linked worktree waits for the first. This comes from our host policy (work.workers = 1), not a tgrep default; it is listed only for completeness.

Suggestions:

• Parallelize the base build. Use N cat-file --batch readers, or a pack-aware reader, with parallel decode and trigram extraction into per-worker sorter runs merged at the end.
• Make first attachment cheaper. Trust Git index stat data (size, mtime, file ID) for paths that are clean at the base commit, and read only dirty or stat-mismatched paths. Alternatively, verify in the background while serving from the base generation plus a dirty-path overlay.
• Report verification progress in status (files verified out of total) so clients can show an ETA.

2. The native watcher advances the input epoch on unmodified files

• Run j.
  • The input epoch went from 613 to 2471 with no commit or content change, rising by about 1,858 during a single 218 s query.
  • ViewQuery::validate refused that query with query-input-changed (tgrep-core/src/managed/views.rs:2316), and the host fell back to a 214.7 s scan. The scan was correct (16 files, same as ripgrep).
  • The same pattern had been served from the index in 2.65 s, 45 s earlier.
• Isolated watcher experiment.
  • While an external tree walk ran intermittently, unmodified files produced attribute-only notifications at about 4/s.
  • Plain reads and idle periods produced 0 events. Last-access time updates are disabled on the volume.
• Cause in the code.
  • notify 7.0.0 maps FILE_ACTION_MODIFIED to Modify(Any) (windows.rs:393-394).
  • watch_event (tgrep-cli/src/serve/shared/managed.rs:645) drops only Access events, so every other event invalidates its path and advances the epoch.

Related: #131 and #151 addressed similar churn in non-managed serve.

Suggestions:

• Before an event advances the input epoch, stat-check the path (size, mtime, file ID) against the last verified state, and coalesce bursts.
• Refuse an in-flight query only when an invalidated path intersects its candidates, not on any epoch change.
• Report watcher event counts by kind, and epoch advances, in status.

3. The scheduler stalls behind unproductive collection or recovery

Run i, under storage pressure:

• 1.85 GB was pending deletion, and 1,119 recovery passes reclaimed 0 bytes.
• No scheduler tick succeeded for about 100 min.
• Status reported collection_in_flight: true while maintenance reported in_flight: 0.
• An invalidated view therefore never reconciled.

schedule_view_with_elapsed (tgrep-cli/src/serve/shared/managed.rs:1617) returns early whenever monitor.pending holds any operation that is not terminal (:1677-1679).

Suggestions:

• Let view reconciliation proceed while collection or recovery is pending, or bound how long a pending operation can block it.
• Keep the scheduler's collection state consistent with maintenance's.
• Back off recovery that reclaims nothing, rather than letting it occupy the only work slot.

4. Query cliffs and the 64 MiB response cap

• -c was exact but took 247 s, against ripgrep's 39 s.
• -l missed a 300 s deadline.
• A pervasive literal and a --glob exclusion query exceeded MAX_RESPONSE_BYTES (64 MiB, tgrep-core/src/managed/mod.rs:91, enforced at tgrep-cli/src/serve/shared/protocol_v2.rs:386) and failed instead of returning output.

Suggestions:

• Short-circuit -l (stop at the first match per file) and -c (count without materializing match records).
• Stream shared responses, or negotiate a larger cap.

5. --json fidelity (observed through the shared daemon)

• Preserve CR in lines.text for CRLF files. It was dropped in 7,190 of 7,276 match records; ripgrep keeps \r\n.
• Emit {"bytes": <base64>} for non-UTF-8 lines, as ripgrep does, instead of lossy text. One such line was affected.