Run BCA and Tree-sitter grammars in-process as sandboxed WebAssembly so function metrics ship in the single static binary

Start with prototypes/structural/worker and prototypes/structural/Dockerfile to understand the native worker and its build environment, then run the proposed tree-sitter-go and tree-sitter-rust spike. Record module size and cold/warm throughput against native on ripgrep and kubernetes. The spike is done when those measurements establish whether the full WASM integration should proceed.

User outcome

dircue map (and the ledger, #97) reports BCA per-function maintainability metrics for 20+ languages by default, from the same single static, cgo-free binary. There's no separately built native worker, no PATH lookups, and nothing to install.

The analyzers run in-process as WebAssembly under a pure-Go runtime (wazero). By construction they have no filesystem, network, clock or randomness access beyond the bytes dircue hands them.

Why

• The mission names BCA as one of the three pillars, but today structural analysis is an explicitly selected separate workflow. Its Rust worker (prototypes/structural/worker: BCA 2.2.0, tree-sitter 0.26, 18 grammar crates) must be built per platform and invoked as a subprocess. That's why #94 left it out of the one-shot map and why #17 and #69 exist.
• A "one-shot" profile that silently lacks function metrics unless the user compiles a Rust worker won't impress anyone. Shipping per-platform native workers multiplies release complexity (#13) and needs a process-isolation policy (#69).
• WASM gives three properties at once:
  • one artifact for every OS and architecture;
  • a capability sandbox: the module physically can't open files or sockets, which strengthens the "never executes anything" story;
  • determinism: no clocks, no randomness, and a fixed memory limit per instance.
• Skeptical systems-minded readers tend to find "20 Tree-sitter grammars plus BCA compiled to WASM, running inside a static Go binary" genuinely interesting. It's also the honest way to leverage BCA rather than re-implement it.

Proposal

• Build BCA plus its grammars for wasm32-wasip1, using a pinned Rust toolchain and wasi-sdk in a hermetic container. The build is reproducible, with checksums recorded in PROVENANCE.json. Embed the compressed module(s) in the binary.
• Host them with wazero, in compiler mode on amd64 and arm64 and interpreter mode elsewhere. Settings:
  • no WASI filesystem mounts, no network, no clock (fixed), no random source (fixed);
  • per-instance memory page limits;
  • context-based cancellation for wall-time budgets (see #1, #2, #91).
• Keep one module instance per worker, reuse it across files, and reset its memory between files. Hand it input bytes through a bounded shared buffer.
• Its output is exactly the native worker's JSON. The native worker remains as a performance option (--set structural.engine=native|wasm) until parity and performance are proven.
• Coverage for unsupported languages, parse errors (clean vs recovered syntax), and per-file time or memory budget hits is explicit, per file.

Tasks

• Spike: compile tree-sitter-go and tree-sitter-rust with BCA to wasip1. Measure module size and cold/warm throughput vs native on ripgrep and kubernetes.
• Full grammar set. Record per-grammar module size and init cost, and choose between one combined module and one per grammar.
• Host integration: instance pool, memory limits, cancellation, bounded I/O buffer, deterministic configuration.
• A reproducible build container and make wasm-analyzers. CI verifies the embedded module checksums, event-driven only.
• Wire it into the map, the ledger and hotspots (#44/#27). Settings in the #91 registry (engine, per-file budget, enabled grammars).
• Budgets: binary-size delta, throughput within an agreed factor of native, peak memory per worker.

Test plan

Unit tests — pkg/structurewasm/ (new package) and pkg/structure/

• WASM host construction: verify wazero runtime configures no WASI filesystem mounts, no network, no clock and no randomness source; assert the import set exactly matches the allowlist.
• Memory-reset between files: send two requests through the same wazero instance and confirm no state leak (same input produces same output; mutating input for request N does not affect request N+1).
• Cancellation: call the WASM engine with a pre-cancelled context.Context and confirm it returns promptly (< 50 ms) with a context.Canceled error, not a hang.
• Budget hit reporting: a pathological fixture (the existing prototypes/structural/tests/fixtures/malformed/Broken.java plus a new synthetic >100 000-line single-function file) must produce status: "partial", not a panic or a timeout without a response.
• Determinism across modes: identical output from wazero compiler mode and interpreter mode on every language fixture.
• Engine flag routing: table-driven tests that structural.engine=wasm routes to the WASM host and structural.engine=native routes to the subprocess; no other values are accepted.
• Fuzz target FuzzWASMHostBoundary in pkg/structurewasm/fuzz_test.go: generate arbitrary (path, language, source) triples; assert no panic, no stuck goroutine, output is valid JSON or a well-formed error. Tie to #87 for corpus persistence.

Independent oracle / parity — tests/structural_wasm/ (new directory)

Tie to #84. The native worker (prototypes/structural/worker) is the oracle; it runs in a container built from prototypes/structural/Dockerfile (the image prototypes/structural/tests/offline.py uses) under --network none to confirm no network dependency.

• Byte-identical parity on all 21 fixtures in tests/structural_breadth/testdata/ (every language, plus unsupported, malformed and generated-Go cases) and on the production 150-file Spring/Roslyn/ASP.NET sample from tests/structure/run.py. Diff must be empty; any deviation is a blocking failure. Record mismatches in the JSONL log.
• Architecture matrix: reuse the matrix already in structural-worker.yml (linux-amd64 on push; linux-amd64/arm64, darwin-arm64/amd64 and windows-amd64 on pull_request and workflow_dispatch), so WASM output is proven identical on every release platform.

Hand-labeled ground truth — tests/structural_wasm/fixtures/ (new directory)

• Committed small fixtures only (< 50 lines each): one per language, covering normal, malformed and unsupported cases. The large Spring/Roslyn files are not committed; they are fetched into .cache/corpus as today (tests/structure/run.py pattern). Tie to #88.
• Reproducible-build checksum fixture: a PROVENANCE.json (following the pattern in internal/jsonschema/PROVENANCE.json) recording the pinned wasm32-wasip1 build toolchain version and SHA-256 of every embedded .wasm module. A new unit test in pkg/structurewasm/ reads the embedded bytes and asserts their hashes match. This test runs on every PR.

Metamorphic invariants — tests/structural_wasm/metamorphic.py (new file)

Tie to #85.

• Appending a blank line to a source file must not change function names, start lines or metrics for any existing function (only end-line of the last function may shift by one).
• Running with one vs eight parallel WASM instances on the same input set must produce identical aggregate output.
• Wazero compiler mode and interpreter mode output are equal on every supported language.

Mutation-testing focus

Tie to #86. Prioritise pkg/structurewasm/host.go (import-allowlist enforcement) and the engine-routing logic in pkg/structure/; both have binary-outcome logic that table tests can miss.

Performance and resource budgets

Tie to #89 and #90.

• Deterministic PR gates (every PR touching pkg/structurewasm/ or pkg/structure/): the binary-size delta ≤ N MB (value set after the spike); per-instance memory page limit configured as agreed; deterministic work counters from #89 (files parsed, bytes handed to the guest, instances created) equal to the committed expectations. Wall time and RSS are noisy on shared runners, so they're reported but not gated there.
• Throughput and RSS budgets (on demand, stable hardware per #90): WASM throughput ≥ 1/3 of native on the 150-file Spring/Roslyn/ASP.NET sample (tests/structure/run.py) and peak RSS per instance within budget.
• On-demand benchmark (make bench-wasm, new target): runs go test -bench BenchmarkWASMEngine in pkg/structurewasm/ against the full Spring/Roslyn corpus and writes a receipt matching the tests/functions/decode-performance/ receipt pattern.

E2E script — tests/structural_wasm/e2e.py (new file)

Runs under workflow_dispatch only (builds are slow). JSONL log fields per repo: command, engine (wasm|native), toolchain_version, wall_seconds, rss_kb, files_parsed, grammar, throughput_files_per_sec, mismatches (list of differing paths). Also emits a human summary table and exits non-zero on any mismatch or budget breach.

CI placement

• Every PR touching pkg/structurewasm/, pkg/structure/ or prototypes/structural/: unit tests, parity on committed small fixtures, binary-size and throughput counters, and the PROVENANCE checksum test. Add a new job wasm-parity to a new structural-wasm.yml (triggers: push, pull_request, workflow_dispatch; no schedule:).
• workflow_dispatch / make wasm-e2e only: the full E2E script, large-corpus parity and the on-demand benchmark.
• The existing structural-worker.yml and structural-prototype.yml are unchanged; the new workflow complements them.

Regression fences

• A test that exercises the import-allowlist immediately fails if a new WASI host function is registered (any addition requires an explicit review decision).
• A test that asserts the embedded module SHA-256 matches PROVENANCE.json fails if the .wasm bytes change without a corresponding PROVENANCE update.

Risks

• Build. C grammar sources under wasi-sdk can hit libc gaps; the spike must retire this risk first.
• Performance. WASM is typically 1.5–3× slower than native. It's acceptable if bounded and parallel, but measure it before defaulting on.
• Binary size. It may grow by several MB; decide the budget explicitly.
• Drift. BCA and grammar updates need the same upstream-tracking discipline as the Enry and go-git forks (#96).

Success criteria

• dircue map includes function metrics by default for every supported language on every release platform, with no extra install.
• Output is byte-identical to native BCA.
• The sandbox is proven by tests.
• Size and throughput budgets are published.

Related: #17, #69, #44, #27, #13, #91, #96. Related idea issues: #97, #107, #110.

Sequencing: Independent; start with the spike. It feeds function rows in #97 and BCA parity in #110 (deferred from #107).

---

This proposal came out of an idea-wizard session on 2026-09-23. It isn't a release commitment, and the milestone is deliberately unset so the owner can decide per idea. It must preserve dircue's contracts:

• deterministic, offline output, with no execution of inspected content;
• the legacy Linguist-compatible CLI and output unchanged;
• unknown never rendered as absent;
• no provider findings or verdicts imported;
• CI event-driven only (no schedule:).