Referee review of the model pages (6 Nov 2026)

Start by reading the model pages and working papers reviewed in the issue at commit a724a13, then compare their status lines and evidence with the listed findings and links. The requested work is to decide which findings to publish now, settle on a shared status vocabulary, and show the provenance of reported bands; done means those editorial decisions have been applied consistently across the model pages.

This is a referee-style read of the overview, methodology and validation pages for each model, plus the working papers, on vahid/r5-ui-models at a724a13. I read them against the 6 Nov round-5 findings. No code or page was changed.

The pages are unusually candid. Identities, anchors and passthroughs are labelled as such. What remains falls into three groups:

1. Findings from the last 48 hours that have not reached the pages.
2. A few status lines that claim more than the evidence below them.
3. A vocabulary ("validated", "replication", "benchmark") that means different things on different pages.

Each model below lists the undisclosed points first, then the claims a referee would reject.

OBR emulator

Not disclosed:

• #42 is now diagnosed, but the page still says the cause is "not yet diagnosed". PIF and GDPMPS are one identity written twice, so nominal GDP drifts with PIF (106.7 → −4.6 → 317). That drift drives FYCPR's 82.77% error. The l.690 nominal-GDP pin makes the household sweep linear to 0.3% over 25 quarters. Every scorecard number is provisional until #42 is decided.
• After q12 the household channel is dominated by noise. A tax cut has the wrong sign at q24, and rise plus cut comes to 125% of the peak.
• Bank Rate's scale has now been tested. The page still says "untested". Against BoE QB 1999 the response is 3–5× too small. Hosted results run on the stale pin (#218); on the branch the response is proportional to within 4% inside the window.

Reject:

• The overview status "Validated for selected scenarios". The page's own registry rates policy counterfactuals "weak", and no quarter converges.
• "~0.34× by q12 vs OBR 0.3". This compares a q12 value with an impact figure. Year 1 is 0.145, about half the OBR figure, and the profile rises where the OBR's tapers.

Links: obr #42, #43, #50, PR #48, #201, #217, #218.

BoE SVAR

Not disclosed:

• The posterior is explosive. 99.2% of production draws have a maximum companion root above 1, and only 4 accepted draws are stable. MP-shock IRFs never revert: per 25bp, CPI is −0.90 at h80. The words "explosive", "unit root" and "stationarity" appear on no page.
• The band fixes have failed. 90% coverage is disclosed (72% at h8). The pages do not say that Gaussian-k, conformal and Lenza–Primiceri recalibrations all fail to reach it (0.82 / 0.82 / 0.81–0.87). That leaves stochastic volatility or fat tails as the remaining route.
• The MP-shock GDP response is 2–4× the BoE 1999 range per pp-quarter, and front-loaded.
• Prior settings are missing. The pages do not give the prior hyperparameters, nor say that θ is off in the coverage evaluation but set to 1 in the published spec.

Reject:

• The status "Validated replication for selected outputs" and "replicates the GDP decomposition". 37.4% against ~40%, with a ±14pp band, is consistent with the paper; it is not a replication.
• "Bands are correct" without saying the word means posterior-correct, not calibrated.

Links: boe #11, #12, PRs #19, #20, #21.

FRB/US

Not disclosed:

• The transfers multiplier is below every matched benchmark (0.28 against 0.36–1.52).
• The 2022–23 tightening counterfactual moves core PCE by only −0.12pp, against about 3pp in FRBSF EL 2025-03.
• The fiscal closure is not stated.
• The Phillips-curve explanation is unbacked. The page attributes the −0.034 against −0.08pp gap to "a flatter current-vintage Phillips curve" without a vintage comparison.

Reject:

• "CBO central ≈0.3" for the tax cut. Whalen & Reichling give no central value; 0.3 is the low end of a no-tightening row.
• The Coenen 0.7–1.0 and 1.1–1.2 bands as written. They are chart readings, and 0.9–1.3 is government investment. The site still carries the labels that PR #12 corrects.

Links: frb #13, #14, #209, #212.

US HANK

Not disclosed:

• The MPC level is never shown. It is 0.075 on the production grid and 0.09–0.10 on a refined grid, against an empirical 0.15–0.25. The cause is the smooth convex illiquid cost.
• The spending multiplier turns negative. At ρ 0.8 the 20-quarter cumulative multiplier is −0.015, and Y is negative from q3 while G is still elevated.
• The sign of that multiplier depends on the MPC calibration. The β-het variant gives +0.309, but it needs β_L of 0.53 a year.
• Monetary π peaks on impact at −1.34pp annualised.
• Missing parameters. The pages do not give the Taylor and Phillips-curve parameters or the per-quarter unit convention.

Reject:

• "A tax-financed spending expansion raising output" (overview). That holds on impact only.
• Implementation fidelity "strong" while vphi misses by 17.3%.

Links: hank #5, #8, PRs #7, #9, #211.

DEFINE-UK

Not disclosed:

• Most clean-room sections are stubs. §3.3.3 and §3.4.1–7 are stubs, so there is no independent solve and no size or sign variant. The paper says milestone 2 is not passed; the validation page does not say so plainly.
• 17 power-sector equations as printed in the manual depart from the oracle. For example, P_ELEC comes out 81% higher, RESTR_PS 22×, and prop_NFF 0.5 against ~0.98. A solve faithful to the manual cannot match the oracle's GDP path.
• Two scenarios break stock-flow consistency (SFC) and are refused. Both are fossil-fuel-ban runs, and the break is an electricity valuation gap.

Reject:

• "Every row and every column … closes" and "each of the 22 published scenarios is implemented".
• "Validated to the ceiling the publications allow". Our missing solver sets that ceiling as much as the publications do.

Links: define #16, #17, PRs #15, #18, #203.

OG-UK (olg/)

Disclosed only in a caption: the showcase table predates the July 2026 labour-tax-rate fix.

Not disclosed:

• The debt sign is a denominator artefact. Debt shows −0.08pp because reform shares are taken over baseline GDP. On the reform's own GDP it is +0.05pp (og PR #4).
• G rises 0.2pp, because the closure spends the revenue.
• GDP rises 0.07% in 2027-28 under a tax rise.
• HF6X includes the BoE. The page compares it with "debt-to-GDP" without stating the model's concept. The ex-BoE series, CPOA, was 87.5% at end-March 2025.
• The 8-sector TPI does not converge.

Links: og #2, #5, #6, PRs #4, #7, #202.

PolicyEngine microsimulation

The page is sound and its gaps are disclosed. Before £6.46bn feeds the OBR bridge, the costing should have a committed scored run and an interval.

Cross-model vocabulary

Page	Status line	What the evidence supports
obr	Validated for selected scenarios	Anchored by construction
svar	Validated replication	Consistent with one figure-read FEVD
define	Validated to the ceiling	Design gates plus one table
us-hank	Replication	Calibrated targets plus 6 of 7 solved parameters
olg	Calibrated, not outcome-validated	Accurate as stated
frb-us	Matches pyfrbus	Code-equivalent to 1e-8

• "Replication" has four meanings: code equivalence, statistical consistency, solved steady-state parameters, and table reproduction.
• "Benchmark" has four meanings: an HMRC sanity check, a naive forecast, multiplier "plausibility bands" (still reported as "inside range"), and a future target.
• "In band" bands are mostly self-set, and DEFINE's growth band was set after the run.
• "Implementation fidelity: strong" is not comparable across models. It covers FRB/US at 1e-8, but also the SVAR with a failed ranking and HANK with vphi 17% off.
• The one cross-model external benchmark is missing. BoE QB 1999 (0.05–0.09% GDP per pp-quarter of a rate rise) appears on no page, yet it shows that no UK model lies in range.

Recommendation: define one status vocabulary on models/ and apply it everywhere:

• code-equivalent
• consistent with published outputs
• calibrated
• anchored
• exact for implemented rules

Drop "validated" from status lines. Print each band's provenance (external, pre-registered or post hoc) next to it. Show cross-model inflation per 100bp side by side (0 / −0.03 / −1.34 annualised / about −2.5% on the level) rather than averaging it (#213).

Decisions for Vahid:

1. Adopt the vocabulary above.
2. Which undisclosed findings go onto the pages now, and which wait for #42, boe #12 and og #5.

🤖 Generated with Claude Code