Data layer: capture a performance baseline before the database migration

Start with the sub-issue's Done-when list and the existing test:e2e:cdp setup (remote debugging on port 9223), then find where the phase boundaries live: the book load path, tokenize, reanchor, and the selector index rebuilds. The sample data is c-sharp/assets/WEB and test-data/pt9-projects. Done means a seeded generator that yields small, medium and large datasets from one command, gated performance marks, a harness that writes JSON from an npm script, and baseline numbers posted on the issue. The open questions on synthetic data and reference hardware need maintainer input first.

[Generated by Claude]

Purpose

Sub-issue of #403. Before any data-layer change, capture a repeatable baseline of the WebView and the extension host, then rerun it at each migration checkpoint on the same machine. The migration moves memory and time from the WebView into the host, so later checkpoints compare both sides against this baseline. This issue changes no data-layer behavior.

Baseline: results on this issue, measured at 5bfb334e with core 0d06e2f7; the JSON is in perf/results/.

Metrics

In the app (perf:app), each with WebView timings:

• First open after the app starts (cold): the packaged app restarts before each run, then the view opens on the tier's book.
• Open the view in a running app (warm): USJ fetch, tokenize, re-anchor, draft fetch and parse, store mount, book render, and open to interactive.
• Switch away and back to the book: to Jude and back, timing the return.
• Edit one gloss: dispatch to render, then the autosave it triggers.
• List saved projects in the picker (getProjectsForSource).
• Save the analysis to its project (saveAnalysis), with the bytes sent.
• Open the concordance: reading and indexing every book of the source.
• Bytes on the wire: the draft (sent on load and by every autosave) and the saved-projects list.
• Heap, WebView and host, after a forced GC: before the view opens, view loaded, full undo history (100 snapshots), after the picker, and after Save. Later readings are deltas against the first, since the host's heap belongs to the whole platform. The host reports heapUsed, external and rss.

In the extension host, beside every WebView call that reaches it (getDraft, saveDraft, saveAnalysis, getProjectsForSource): the host's time, split into storage read, parse, shard compare, storage write and reply serialize. Draft fetch's shard compare is the re-serializing of each shard into the compare cache. Where reads and parses interleave (a draft's shards, the saved projects), read is the span of the reads less the parsing within it. Transport is the WebView's time for a call minus the host's; the two processes' clocks have different origins, so only durations are compared.

In Node (perf:bench), at every tier including large: the load steps, splitting a draft into shards and merging them back, committing a gloss and rerunning the selectors, serializing the draft, building the concordance, and the heap held by a parsed draft and a full undo history.

Out of scope: page-fetch latency, whose marks come with the work that adds page reads, and a CI regression threshold for the bench.

Dataset

WEB, which paranext-core ships as a sample project, glossed by a seeded generator (perf/dataset/, seed 407). Each draft has stale, competing, phrase (some spanning segments), free-translation and morpheme records, payloads shared by content, and saved projects for the picker. Generation fails unless the draft has no invariant violations and re-anchoring leaves it unchanged, so loading one never writes.

Tier	Text	Coverage	Saved projects	In the app
small	Philippians	100%	5	yes
medium	Psalms	100%	5	yes
large	whole Bible	50%	5	no: its 133 MB draft is over core's 100 MB websocket message cap
large-loadable	whole Bible	30%	1	yes: the largest draft one message carries

Known bias, accepted. Against a real 38-book PT9 import, senses per form, gloss length and phrase density match, but WEB's English has fewer word forms: about 2.8× the links per token payload. Costs that scale with payload count, such as the pool index and catalog rows, read 2–3× low; costs that scale with link count are comparable. The summaries note this beside the numbers. Having the generator split forms toward a links-per-payload target was rejected because it would push payloads per surface form from 1.4 to about 3.9. npm run perf:shape compares a tier's shape with a real project.

Tooling

• Marks. src/utils/perf-marks.ts records ilz: performance entries and is a no-op until turned on: in the WebView by the interlinearizer.perfMarks localStorage key, in the host by the interlinearizer.setPerfMarks command. interlinearizer.readPerf returns the host's entries and its process id.
• perf:app drives a production build of the extension in the app over CDP and turns both sets of marks on. The WebView's GC is forced through the app's remote debugging port. A packaged app strips NODE_OPTIONS, so --expose-gc can't reach the host; the run instead opens the host's Node inspector (process._debugProcess) on port 9229 and forces the GC through it.
• perf:bench times the pure functions under node --expose-gc.
• Results. Each run writes one JSON to perf/results/, keyed by scenario with webView and host sections and recording the commit, core commit, machine and load average. npm run perf:summary renders one table per scenario, with the host's time and steps as ↳ rows under the WebView call they serve.
• Chrome DevTools and React Profiler stay for explaining numbers, not producing them.

Protocol

• Reference machine: @alex-rawlings-yyc's Windows desktop (i7-12700KF, 20 threads, 34 GB). Every checkpoint runs on it, idle.
• App: a packaged core, cold and warm, with a production build of the extension.
• Runs: 10 per scenario, in the app and the bench, reported as median and p90; at 10 runs, p95 would only be the slowest run.
• Commit: a fixed, clean commit; results files don't count toward the dirty flag.

Rerunning at a checkpoint

1. Build a packaged core (npm run package in paranext-core) at the baseline's core commit unless the checkpoint needs a newer one, and set PERF_APP_PATH to its executable.
2. Check out the checkpoint commit with a clean tree, and keep perf/dataset/tiers.ts and the seed unchanged so every checkpoint measures the same data.
3. Free ports 9223 and 9229, then run npm run perf:datasets, npm run perf:bench and npm run perf:app. e2e-tests/README.md has the details.
4. Commit the two results JSONs and post npm run perf:summary's output here, naming the checkpoint.

Done when

All of this is needed before the migration starts:

• The harness, datasets, and WebView and host marks are merged (#411), including the medium tier's Save flake fix
• Host marks at the command phase boundaries, off by default
• Host heap readings at the WebView heap points, plus after the picker and after Save
• One results JSON and one summary per run, with the host beside the WebView for every app scenario
• Dataset bias and cross-segment phrase share settled
• Baseline run at a fixed commit on the reference machine, 10 runs per scenario, cold and warm on a packaged app, with the results JSON committed and the summary posted here and linked from #403