Complete the parity atlas: BCA per-function parity, large corpus, informational comparisons, and HTML summary

Start by reading the existing differential runner in tests/atlas/ and its e2e.py entry point, then review the oracle setup in tests/conformance and the reproduction guide. The issue spans BCA classification, corpus freshness, informational comparisons, HTML rendering in tests/atlas/render_summary.py, fixtures, and workflow integration. Done means the selected scope is reproducible, tested, documented, and preserves the stated opt-in and non-gating policies.

Context

This is the continuation of #107, which ships the first installment of the parity and accuracy atlas in 1.0.0 (~50-repo corpus, Linguist 9.7.0 exact parity, per-file scc parity, accuracy cards, on-demand publication). The work below was explicitly deferred from #107's 1.0.0 scope and should build directly on its differential runner (tests/atlas/), corpus pins, and oracle containers.

Deferred scope

• BCA per-function parity. Compare dircue's structural output against the standalone BCA CLI at the pinned version, per function: cyclomatic complexity, cognitive complexity, Halstead measures, LOC, and maintainability index. Requires #98 (BCA as an in-process WASM module) or a hermetic oracle container running the native Rust worker from prototypes/structural/worker.
• Scale to hundreds of repositories. Expand the atlas corpus from ~50 to several hundred well-known repositories across ecosystems, including long-tail language coverage, very large monorepos, and non-source-heavy repositories.
• tokei/cloc informational comparisons. Run tokei and cloc on the same corpus and document per-language disagreements. These are informational only: their language definitions differ from both Linguist and scc, so no identity gate is applied. Disagreements are explained and categorized, not suppressed.
• Static HTML summary page. Reuse the technology from #104 to render atlas results as a static HTML page published as a release asset, replacing the Markdown summary written by tests/atlas/e2e.py.

Proposal

• Extend the oracle container set (#84) with a BCA container at the pinned version. Run bca --json on each corpus entry and compare per-function output with the structural worker's output. Where dircue marks a result partial or unknown, assert that the oracle fact falls inside that declared scope per #84's soundness and honesty checks.
• Scale the corpus incrementally: add entries in batches, validate oracle results before committing pins, and keep the (repo-commit-sha, tool-version) cache from #107 so existing pins are never re-fetched.
• Add tokei and cloc steps to tests/atlas/e2e.py; log per-language counts alongside the existing oracle columns; write a separate informational-comparisons.md section in the atlas summary.
• Replace the Markdown summary with an HTML page generated by a new tests/atlas/render_summary.py (new), reusing the static-site components from #104.

Tasks

• BCA oracle container, pinned and reproducible, extending tests/conformance/tests/atlas oracle infrastructure (#84)
• Per-function diff classification: matched, bca-disagreement, known-structural-gap, error; soundness and honesty assertions per #84
• Corpus expansion policy and automated freshness checks: a script that flags corpus entries whose pinned commit is more than N months behind HEAD on the default branch
• tokei and cloc harness steps in tests/atlas/e2e.py (informational, non-gating)
• Static HTML summary from tests/atlas/render_summary.py (new), attached as a release asset
• Updated reproduction guide in tests/atlas/REPRODUCING.md

Test plan

Unit tests

• Per-function diff classification: unit tests in tests/atlas/ for each outcome category, including the boundary case where dircue reports partial and the BCA oracle has data (honesty assertion, not a bug).
• HTML renderer: snapshot tests comparing generated HTML against committed expected output; at least one fixture with a known discrepancy.
• Corpus-staleness check: a test that the freshness script correctly flags a synthetic entry whose commit is expired and passes a current one.

Independent oracles (#84)

• BCA oracle container at the version matching prototypes/structural/worker/Cargo.toml's BCA pin; --network none; per-function JSON output.
• tokei and cloc at pinned versions; informational only; logged but not gated.
• Soundness assertion: every fact dircue marks complete is confirmed by the BCA oracle; every oracle fact dircue lacks is inside a partial or unknown scope.

Hand-labeled ground truth and fixtures (#75, #88)

• Extend the tests/atlas/labels/ per-question JSONL files to the larger corpus, using the same format as #107.
• Track precision and recall on the new corpus entries; update docs/ACCURACY.md and the embedded binary data.
• No large binary corpus data in git; manifests and hashes only, per #88.

Metamorphic invariants (#85)

• Extend the #107 idempotency and subset invariants to cover the BCA oracle and the larger corpus.
• Inject a synthetic function with a known cyclomatic complexity mismatch; verify it appears as a classified discrepancy in output.
• tokei/cloc step: running the informational comparison on an identical input set produces identical output on successive runs.

Fuzz targets (#87)

• FuzzBCADiffClassify (new): seeds from recorded per-function BCA oracle rows; property: every input returns a non-empty, known-vocabulary category label.
• On-demand only; persisted corpora in .cache/fuzz/.

Mutation-testing focus (#86)

• BCA per-function counter comparison: boundary mutations on cyclomatic and cognitive complexity values.
• Soundness assertion: a mutation that silently promotes a partial to complete must be caught by the honesty assertion.

Performance and resource checks (#89, #90)

• The full corpus run is on-demand only; make atlas-bench (from #107) extended to include BCA timing.
• No schedule trigger; results appended to the existing benchmark history per #90.

E2E script

• tests/atlas/e2e.py extended with --bca and --informational flags; the BCA step is off by default until #98 or the oracle container is ready.
• JSONL log fields extended with: bca_tool_version, bca_total_functions, bca_matched, bca_mismatched, tokei_version, cloc_version.
• Exit policy unchanged: exit 0 when all mismatches are categorized.

CI placement

• BCA and larger corpus: workflow_dispatch-only, added to the existing atlas.yml workflow from #107 as opt-in flags; no schedule:.
• tokei/cloc: workflow_dispatch-only; results informational in the artifact; never block a PR.
• HTML summary: generated as part of the on-demand atlas run and attached alongside the JSONL output.

Regression fences

• Carry forward the Linguist (#72) and Git delta-reader (#73) regression fences from #107.
• Add a BCA regression fence for the first function-metrics discrepancy found and resolved.

Risks

• BCA readiness. This work is gated on #98 landing or a usable oracle container for the native Rust worker. Until one of those is available, BCA steps in the runner exist but are skipped by default.
• Compute at scale. Hundreds of repos × 3+ oracles is heavy. Cache unchanged pins aggressively; run only on demand on stable hardware (#90).
• Labeling effort at scale. Label cost grows with the corpus. Budget time per batch; precision and recall targets may need adjustment as the corpus becomes more heterogeneous.

Success criteria

• A public atlas covering Linguist parity, scc parity, and BCA per-function parity across hundreds of repositories.
• tokei/cloc disagreements documented per language, with root causes.
• A static HTML summary page attached to every tagged release.
• Every number reproducible with a documented command.

Related

#107, #75, #84, #85, #88, #90, #92, #12, #97, #98, #104, #105.

---

Continuation of the work introduced in #107. Milestone unset; the owner decides scope and timing per idea. The same dircue product contracts apply: deterministic, offline output; Linguist-compatible CLI unchanged; unknown never absent; no findings or verdicts imported; CI event-driven only (no schedule:).