Evaluate fff for repeated directory exploration and scanning techniques

Start by reviewing dircue’s existing file enumeration and worker pool, then inspect fff’s pinned APIs for complete enumeration, errors, cancellation, filtering, and index readiness. Establish a baseline for a concrete workload and compare end-to-end cost, memory, correctness, and release impact across the evaluation plan. Done means recording a supported adopt, adapt, or defer decision with reproducible measurements and semantic differences.

Purpose

Evaluate whether fff could improve dircue's directory exploration or scanning implementation. This is future research, not a commitment to add a dependency, persistent service, or new command.

Dircue should remain useful for a single offline invocation against an arbitrary directory, with explicit coverage and reproducible observations. Any integration must preserve the existing language-only path and Git-free operation.

Why revisit fff

FFF is a Rust file-search engine with a resident index, content caching, fuzzy path search, exact/regex content search, and filesystem watching. Its design primarily targets repeated queries against the same directory. Its published search performance claims do not establish a speedup for dircue's complete profiling workload. See the upstream design explanation.

Potential uses:

• Exploration after profiling: help people and agents navigate from a project or relationship to relevant source files and declarations. Approximate filename search could accommodate unfamiliar naming without changing the underlying report.
• An explicit persistent session: amortize enumeration and search across many questions against the same directory, if a concrete use case justifies the additional lifecycle and freshness rules.
• Implementation ideas: study compact path storage, parallel traversal, and batched matching. Improvements to dircue may be worthwhile even if adopting the whole engine is not.

These are hypotheses to test, not implemented capabilities or measured benefits.

Fit and integration constraints

Dircue already enumerates selected files within its Go process and feeds a worker pool. It does not repeatedly spawn find or grep to perform its ordinary scan. A second index could therefore duplicate work and increase startup time or memory consumption.

Search must remain separate from evidence semantics:

• Fuzzy matches and ranking must never establish dependencies, project ownership, or build relationships.
• Ignore policies, indexing exclusions, pagination, time budgets, and oversized/binary-file handling must not silently change inventory totals or structural coverage.
• Completion and freshness must be explicit. A live filesystem index cannot be treated as an immutable Git-revision inventory, or as a consistent filesystem snapshot merely because a watcher is running.
• Access history and query frequency may influence interactive ranking, but must not influence deterministic profiling facts. Persistence and watching should be deliberate choices, with bounded resource use.

Upstream documents Go integration through its C ABI using cgo. Evaluate that cost against dircue's CGO-disabled Go core and cross-platform release process. An optional native helper, potentially using our existing worker distribution approach, is another option to assess; do not assume it is cheaper or appropriate without measurement.

Evaluation plan

1. Select a concrete workload: complete one-shot profiling, repeated directory queries, or a specific traversal/matching bottleneck. Establish a dircue baseline and confirm what dominates its cost.
2. Pin the upstream revision and relevant build features. Inspect the APIs for complete enumeration, errors, cancellation, filtering, and index readiness before designing an adapter.
3. Prototype outside the default execution path. Compare direct integration, a helper, and adopting individual techniques where applicable.
4. Measure cold startup, initial index construction, end-to-end profiling time, repeated-query latency, peak memory, and any persistent disk footprint. Include small trees, large Java/.NET repositories, data-heavy directories, and vendor/generated content.
5. Check correctness separately: exact selected paths, file/byte counts, language JSON, evidence attribution, and coverage qualifications. Exercise Git-free trees, ignored files, symlinks, permissions, concurrent changes, and cancellation.
6. Evaluate release impact on Linux, macOS, Windows, and the existing wheels. Define resource controls and cache invalidation before accepting any persistent mode.

Completion criteria

Record a supported decision: adopt a narrowly scoped integration, adopt individual techniques, or defer/reject it. Include reproducible measurements, semantic differences, resource costs, and distribution implications. Warm search latency alone is insufficient evidence for changing the one-shot scanner.

If a useful feature emerges, create a separate implementation issue with a clear user contract and regression gates. Do not introduce an automatic behavior change based on whether fff happens to be installed.