Forest mode: archive listing, opaque bundles, non-Git VCS roots, and dirty/untracked summaries

Start by reading the forest foundation from #100, then inspect the proposed tests in pkg/forest/archive_test.go, opaque_test.go, vcs_test.go, and dirty_test.go. Review tests/forest/fixtures/ and tests/forest/e2e.py to understand the expected output and oracle scenarios. Done means the four capability areas and forest summary are covered without executing inspected content, with the stated partial statuses and limits.

User outcome

A scan of an old drive or archive directory shows:

• the member list and language composition of zip, tar, jar, whl and Docker/OCI image archives, without extracting them to disk;
• opaque bundles (macOS app bundles, photo and music libraries, VM disk images, SQLite databases) identified and sized, not walked;
• Mercurial and SVN working copies recognized and counted as roots alongside Git repos;
• each Git root's dirty/untracked state summarized (N modified files, M untracked files, by path prefix) without executing git status.

Why

• #100's 1.0.0 scope delivers hostile-FS conformance, nested Git root discovery, forest document shape, and environment-tree summarization. Those are the highest-priority items and the ones that unblock the forest demo.
• The remaining four capabilities complete the "forgotten drive" story but each carries meaningful implementation risk or surface area that is best landed after the 1.0.0 milestone is stable.
• Archive reading is a large parsing surface that needs its own fuzz coverage (#87) before shipping.
• Non-Git VCS (Mercurial, SVN) requires recognizing working-copy metadata formats that dircue has never touched; the risk of silent mis-identification warrants a separate focused delivery.
• Dirty/untracked summarization without executing git requires index parsing (go-git supports this) but the semantics are subtle; a separate issue keeps the scope honest.

Proposal

Bounded archive listing and classification

• List members of zip/jar/whl/nupkg, tar (plain, gz, xz, zst, bz2), 7z headers where a pure-Go reader exists, and Docker/OCI image layouts.
• Classify members' languages and roles from their names and bounded prefixes, recursing to a configurable depth.
• Bomb limits: configurable compression ratio, member count, total decompressed bytes, and depth; any exceeded limit sets status: partial with a bomb_limit reason.
• Report archive content as a separate population, never mixed with on-disk files. #58 remains the owner of artifact-internal metadata.

Opaque-bundle recognizers

• Recognize macOS .app, .framework, .xcassets bundles (Info.plist present), photo and music libraries (.photoslibrary, .musiclibrary), VM disk images (.vmdk, .vhd, .qcow2), SQLite databases, and CoreData stores as typed opaque nodes.
• Report type name, size, and a single-line description; never walk their internal structure.

Non-Git VCS roots

• Mercurial: .hg/ directory present → record root path, hgrc remote URLs (credential-stripped), tip revision if readable from .hg/store/00changelog.d without executing hg.
• SVN: .svn/ directory with entries or wc.db present → record root path and repository URL from wc.db (SQLite, read as data).
• Both: recognized and counted in the forest summary; never parsed deeply; status: partial with vcs_not_git reason.

Dirty/untracked summaries

• Parse the Git index file (.git/index) using go-git's index reader.
• Compare index entries against the commit tree to find modified and deleted tracked files.
• List untracked entries by finding files in the worktree not recorded in the index (bounded to 10k entries).
• Report: modified count, deleted count, untracked count, and up to 20 representative paths per category.
• status: partial with index_parse_failed when the index is unreadable or the format version is unsupported.

Tasks

• Archive listing and classification adapters (zip, tar, 7z, OCI), with bomb limits and a nested-depth setting.
• Opaque-bundle recognizers.
• Non-Git VCS root recognizers (Mercurial, SVN), read-only.
• Dirty/untracked summarizer using go-git index reader.
• Forest summary text: roots table including non-Git roots, storage composition, explained bytes (#97), top unexplained formats.

Test plan

• Unit tests (pkg/forest/archive_test.go, pkg/forest/opaque_test.go, pkg/forest/vcs_test.go, pkg/forest/dirty_test.go, all new): member listing for each archive type; bomb-limit rejection; opaque-bundle detection for each type; Mercurial and SVN root detection from synthetic .hg/ and .svn/ directories; dirty summary for a synthetic index with one modified and one deleted file.
• Independent oracle: hermetic container with unzip, tar, 7z (from p7zip), svn, hg installed; archive member lists compared against dircue's output; SVN working copy checked against svn info; Hg root checked against hg log -l1.
• Fuzz targets: FuzzArchiveReader per format (zip, tar, 7z), seeded with valid small archives; property: never panics, member count matches oracle within bomb limits. Persisted corpora per #87.
• Fixtures: tests/forest/fixtures/ (new, from #100's 1.0.0 scope work); extend with: a zip bomb (100k-member, blocked at limit), a tar with path-traversal names (names reported confined, no extraction), an .app bundle, a minimal .hg working copy, a minimal .svn working copy, a Git repo with one modified and two untracked files in the index. Binaries generated at test time, only generator scripts committed.
• Metamorphic invariants: archive member count equals oracle count (within bomb limit); opaque-bundle size equals du -sb output; non-Git root appears exactly once in the forest even when nested inside a Git repo; dirty count is zero for a clean index.
• E2E: extend tests/forest/e2e.py (from #100's 1.0.0 work) with additional fixture scenarios; log archive_members, opaque_nodes, vcs_roots_non_git, dirty_files to the JSONL output.
• CI placement: unit tests in existing test job every PR; oracle harness workflow_dispatch only; archive fuzz targets on-demand per #87.

Risks

• Archive parsing surface. Each format is a new parsing dependency; fuzz each one before shipping.
• Non-Git VCS. Mercurial and SVN working-copy formats are under-documented; mark coverage partial conservatively and never claim a dirty count is exact.
• Dirty/untracked cost. Walking the worktree to find untracked files is O(worktree size). Apply the same budget limits as the main scan.

Success criteria

• A zip archive's member list matches the oracle for all corpus samples.
• A macOS .app bundle is recognized as an opaque node, not walked.
• A Mercurial and an SVN working copy appear as distinct roots in the forest summary.
• A Git repo's dirty summary matches git status --short output for all corpus repos (within declared partial-coverage exceptions).
• No archive, bundle or VCS file is executed at any point.

Related

• #100 (parent; 1.0.0 scope delivers the forest foundation this builds on)
• #101 (content identity for directory roots)
• #58 (artifact-internal metadata; archives are in scope here for member listing only)
• #87 (on-demand fuzzing for archive readers)
• #97 (explained-bytes ledger)

---

This follow-up came out of an idea-wizard session on 2026-09-23. The milestone is deliberately unset. It must preserve dircue's contracts: deterministic, offline output, with no execution of inspected content; the legacy Linguist-compatible CLI and output unchanged; unknown never rendered as absent; no provider findings or verdicts imported; CI event-driven only (no schedule:).