[Bug]: fff 0.9.4 watcher keeps rescanning large non-git workspaces and holds the backend at 60–190% CPU

从 apps/server 开始，找到 @ff-labs/fff-node 的 pin，并检查 issue 中的版本比较和 bench.cjs harness。使用该 harness 或等效的复现方式，在大型非 Git workspace 上验证依赖项更新。完成的标准是项目使用 fff-node 0.11.0，且重新扫描频率和 CPU 使用率得到改善，同时不添加上游的二进制嗅探变更。

Before submitting

• I searched existing issues and did not find a duplicate.
• I included enough detail to reproduce or investigate the problem.

Area

apps/server

Steps to reproduce

1. On macOS, add a large non-git folder as a project. Mine is ~/code: the parent folder of about 170 repos, about 1.65M files in total. fff indexes about 120k–225k of them after its built-in skips.
2. Keep agents working in repos under that folder, so files change all the time: builds, installs, logs and scratch output.
3. Use anything that builds the path index for that project once (the @ picker, the Files panel or a file link). Then leave the app open.

Expected behavior

After its first scan, the workspace index follows changes cheaply. An idle app does not keep 1–2 cores busy.

Actual behavior

The desktop main process (which hosts the backend) stays at 60–190% CPU with nothing running. sample shows the time inside fff's native threads, not in JS. The Node main thread is idle in kevent in 5151 of 5922 samples.

fff-scan (busy in 100% of samples):
  fff_search::scan::ScanJob::run
    -> bigram_filter::sniff_binary_for_non_indexable
      -> FileItem::detect_binary_per_byte -> read()   4229
      -> FileItem::detect_binary_per_byte -> open()   1549

fff-watcher-own (busy in ~50% of samples):
  FilePicker::handle_file_modify
    -> FileItem::detect_binary_per_byte -> open()     1308
    -> FileItem::detect_binary_per_byte -> read()     1252

A new fff-scan thread appears about every 35–40 s, so fff keeps rescanning the whole tree. server.trace.ndjson shows only two WorkspaceSearchIndex.make spans in that window, each about 2.8 s and successful. So T3 is not rebuilding the index. The rescans come from inside fff.

Cause

T3 pins @ff-labs/fff-node 0.9.4. Its watcher (crates/fff-core/src/background_watcher.rs) starts a full rescan whenever any of these happens:

• a debounced batch has more than 1024 affected paths (MAX_OVERFLOW_FILES), counted before ignore filtering
• more than 1024 new files are waiting in the overflow list
• FSEvents sets its rescan flag
• any .gitignore or .ignore changes anywhere under the root

None of these are rate-limited. Under a folder that holds many active repos, one of them fires almost all the time.

fff fixed this upstream in dmtrKovalenko/fff#751, "Reduce amount of rescans in giant /Users/neogoose like folders" (commit 031005e). It adds a RescanThrottle: watcher rescans wait at least 30 s, and 5 minutes for non-git roots with 1M or more files. The first stable release with the throttle is 0.10.3. The current release is 0.11.0.

A second cost remains in 0.11.0 and on fff main. FileItem::detect_binary_per_byte reads every new or changed file to EOF, and it keeps reading after it has found a NUL byte. On the watcher path (handle_create_or_modify, add_new_file) there is no size cap. So each append to a log, sqlite file or media file under the root re-reads the whole file. That belongs upstream in fff.

Measurements

I ran a standalone harness with the exact options T3 passes for the paths index (disableMmapCache, disableContentIndexing, aiMode: false, enableFsRootScanning, enableHomeDirScanning). It ran on the same ~/code with real agent activity. Every 20 s it also writes and deletes 1500 files under a node_modules folder. Each version ran for about 4 minutes, one after the other, on an Apple M4.

The bump removes about 40% of the CPU and about half of the rescans. Most of what's left is the per-file sniff described above.

const fs = require("node:fs");
const path = require("node:path");
const [fffDir, root, seconds] = process.argv.slice(2);
const { FileFinder } = require(path.resolve(fffDir, "node_modules/@ff-labs/fff-node"));
const finder = FileFinder.create({
  basePath: root, disableMmapCache: true, disableContentIndexing: true,
  aiMode: false, enableFsRootScanning: true, enableHomeDirScanning: true,
}).value;
const churnDir = path.join(root, ".fffbench-churn", "node_modules", "pkg");
const cpuStart = process.cpuUsage();
let scanning = finder.isScanning(), scans = scanning ? 1 : 0;
setInterval(() => { const now = finder.isScanning(); if (now && !scanning) scans++; scanning = now; }, 100);
setInterval(() => {
  fs.mkdirSync(churnDir, { recursive: true });
  for (let i = 0; i < 1500; i++) fs.writeFileSync(path.join(churnDir, `f${i}.js`), "x");
  for (let i = 0; i < 1500; i++) fs.unlinkSync(path.join(churnDir, `f${i}.js`));
}, 20_000);
setTimeout(() => {
  const cpu = process.cpuUsage(cpuStart);
  console.log({ scans, cpuSeconds: (cpu.user + cpu.system) / 1e6 });
  process.exit(0);
}, Number(seconds) * 1000);

Suggested fixes

1. Bump @ff-labs/fff-node to 0.11.0 to get the rescan throttle. I'll open a PR that links this issue.
2. Upstream in fff: stop detect_binary_per_byte at the first NUL byte, and cap or skip large files on the watcher path, as the bulk scan already does with MAX_FFFILE_SIZE.
3. In T3, consider not keeping a live watcher on very large non-git roots, or on a root that contains other registered projects.

Related

• #11306: the same fff threads block the event loop on a home-directory project.
• #10917: the 15 s index scan timeout.
• #10918: no way to exclude directories outside git.
• #11500: keeps a partial index on timeout. That PR is complementary and doesn't touch the watcher.

Impact

Major degradation or frequent failure

Version or commit

T3 Code (Nightly) 0.0.45-nightly.20260930.2493. Code is current main (8630e1ac7).

Environment

macOS 26.6.2 (25G83), MacBook Air M4 (Mac16,12), 16 GB RAM, local APFS disk.

Workaround

Avoid adding a large non-git parent folder as a project, or avoid opening anything that indexes it. The index closes 15 minutes after its last use.