UK national calibration reads stored input columns at the frame period: 76 £ rows compare 2024 values with 2025 targets

Start with _prepared_column_values, _expression and _column in target_materialization.py, then trace stored-column handling in uk_runtime/ledger_targets.py and the self-employment route in uk_runtime/measure_simulation.py. Reproduce the period mismatch with the engine-free routing tests described in the issue, including frame-period reads and loss relief. Done means stored values are measured at the declared period or computed at the calibration period, with intended frame-period reads explicit and the affected calibration rows checked.

Summary

The UK national calibration measures every target at the 2025 calibration period, except where the target's value variable is a stored frame column. Those rows read the stored column as it is, at the frame's own period (2024), and compare it with a 2025 target. The engine-computed rows are measured at 2025, as intended.

On the national file built at 810315ac2, 76 £ rows are affected without intent:

• 73 SPI component amounts (employment, dividends, property, savings interest, private pensions and self-employment, by total-income band);
• the ONS household road-fuel row;
• OBR statutory maternity pay;
• DWP workplace-pension employer contributions.

Calibration reports these rows as hit, or as smaller or larger misses than they are. At the engine's own 2025 values, road fuel is 6.9% low, the SPI dividend, property and savings amounts are about 4.3% high, private pensions 2.3% high, SMP 3.5% high and workplace-pension employer contributions 4.1% high. The other 365 £ rows are engine-computed at 2025 and period-consistent.

No issue tracks this. #1278 holds only the road-fuel remeasure.

National weighted aggregates and published statistics only. Paths are relative to packages/microcosm-build/src/microcosm/build/. Line numbers are at main 41209b38c. target_materialization.py and uk_runtime/measure_simulation.py are byte-identical at 810315ac2, and the cited lines of uk_runtime/ledger_targets.py are unchanged.

Mechanism

The frame stores its inputs at 2024. The 810315ac2 national run's measure receipt records input_period: "2024" and calibration_period: 2025.

1. Plain-column bindings read the frame with no period. A binding with no kind is prepared by _prepared_column_values (target_materialization.py:867-878). It reads value_variable through _column (:1001-1005), or each term of a value_expression through _expression (:988-998), which also calls _column. None of the three takes a period.
2. The UK adapter returns the raw frame column. UKFrameTargetAdapter.column (uk_runtime/ledger_targets.py:1617-1629) returns the stored table column. It raises the missing-column KeyError only for a variable that isn't in the frame.
3. Only missing variables reach the engine. resolve_target_measures sends a skip to the measure provider only when its reason parses as a missing column (target_materialization.py:265, provider.compute at :297). The provider's fallback is simulation.calculate(variable, year) at the calibration year (uk_runtime/measure_simulation.py:428). A stored column never raises, so it is never computed at 2025.
4. measurement_period doesn't reach the column path. Only provider kinds receive it (target_materialization.py:688, :708). On a plain-column binding it disables the pre-existing-measure guard at :668-673 and does nothing else, so declaring it would not fix these rows. Today only hmrc.cgt.liability_total declares it on a plain-column binding, and that binding reads a reserved dated name.
5. The self-employment rows take a second route. self_employment_income_after_loss_relief is a provider measure, but its branch reads the frame's stored self_employment_income when that column is present (measure_simulation.py:359-379; the comment at :366 says so).

The code already guards this risk in two places:

• CGT. The reserved CGT names exist because "using the model's input name here would let a persisted base-period column bypass requested-year calculation" (measure_simulation.py:74-85).
• Persisted measures. The resolver refuses persisted CGT, bus-boarding and high-value-home measures (:781-806).
• The guard's own comment. It notes that "a pre-existing column has no period provenance and may be stale" (target_materialization.py:685).

The plain-column path has no such guard.

One consequence is that a single row can mix periods. The SPI amount rows band people on total_income_after_loss_relief, which the provider computes at 2025, but they sum the stored 2024 amount.

Which rows

Of the 451 £ rows, 86 read stored columns:

• The 76 above. Each row's error equals the engine's 2024→2025 uprating index for that column.
• Ten bus and rail rows.
  • Seven are FY2024-25 facts under reviewed_no_index holds (uk/uprating_holds.json), so reading them against the 2024 frame matches their period.
  • The fare-receipt rows have no hold. The two England fare-receipt rows read 2024 bus_fare_spending, which the engine CPI-uprates by 3.4% into 2025.
  • These ten are left out of the 76 and need a verdict in the fix.

The other 365 £ rows are engine-computed: their final_estimate equals the engine's own 2025 value to within 0.01%.

Diagnostic (excluded) rows were not audited. Any that reads a stored column carries the same offset.

Measured effect

Measured on the 810315ac2 national file:

• Every tested row's final_estimate (113 rows) is reproduced to within 0.01% from the stored columns at 2024 and the engine variables at 2025.
• The engine's 2025 values are the same under policyengine-uk 2.132.6 (the build engine) and 2.130.1 (the runtime pin). The fuel projection is unchanged to 2.137.

Rows	Target rows	Final vs target (as calibration reports it)	Engine 2025 vs target	Engine 2024→2025 index
SPI employment income amounts	12	−7.1%	−2.2%	+5.3% (average earnings)
SPI dividends	11	−0.1%	+4.3%	+4.4% (per-capita GDP)
SPI property income	13	0.0%	+4.4%	+4.4% (per-capita GDP)
SPI savings interest	13	−0.1%	+4.3%	+4.4% (per-capita GDP)
SPI private pensions	12	−1.2%	+2.3%	+3.6%
SPI self-employment, after loss relief	12	+0.7%	+1.4%	+0.7% (mixed income)
ONS road fuel, COICOP 07.2.2	1	0.0%	−6.9%	petrol −7.1%, diesel −6.5%
OBR statutory maternity pay	1	+0.1%	+3.5%	+3.4% (CPI)
DWP workplace-pension employer contributions	1	0.0%	+4.1%	+5.3% on employer contributions, 0% on salary sacrifice

In £:

• Road fuel. ONS CY2025 is £35.10bn and the engine's 2025 value £32.67bn. Of the −6.9%:
  • about 5.5% is the period error, since ONS's CY2024 row (£37.03bn) is that much above CY2025;
  • about 1.6pp is the engine projecting fuel spend down faster than ONS measured.
• Dividends. £61.55bn target against £64.20bn at the engine period.
• Savings interest. £21.87bn against £22.81bn.
• Workplace-pension employer contributions. £104.2bn against about £108.5bn.

The employment rows show that the period error can hide which misses are real:

• Calibration reports the 12 bands 7.1% short. At the engine period they are 2.2% short.
• Bands from £30k up move from −5.9% to −0.9%.
• Below £30k a real shortfall of about 7.5% remains.

The calibration diagnostics (final_estimate, fraction_within_10pct), and anything that reads them, carry the same measurement. The road-fuel row, for example, is reported as hit (−0.002%).

Why it matters for Budget measures

Static costings scale with the base at the engine period, which is the period the runtime uses:

• Fuel duty and VAT on fuel. The household road-fuel base is 6.9% low, so a duty change is costed about 7% too small.
• Dividend, savings and property income. The bases are about 4.3% high. This affects rate changes, the dividend and personal savings allowances, the savings starting band, and freezes that act on them.
• Pensioner income tax. Private pension income is 2.3% high.
• Salary-sacrifice and pension-relief measures. The employer-contribution base is 4.1% high. Separately, policyengine-uk does not uprate pension_contributions_via_salary_sacrifice, so that part stays at 2024 cash. That is an engine issue, outside this one.
• SMP. 3.5% high; small.
• Employment income and NICs. The SPI rows ask the solve for about 5.3% more employment income than the engine has at 2025. This may explain part of the OBR employee-NICs overshoot (+3.6%). That is untested: it needs a re-solve.

The weights also move. The solve fits 2024 values to 2025 levels, so it moves weight towards households that carry these incomes, and away from fuel buyers, by up to a year's growth. Every table that correlates with them moves too. Fixing this changes the solve, and every baseline with it.

Proposed fix

1. Measure stored columns at the measurement period.

• Pass the period. _prepared_column_values, _expression and _column take the binding's measurement period: measurement_period, or else the calibration period.
• Route stored columns to the provider. When that period differs from the frame's time_period, the UK adapter treats a stored source column as missing. The resolution loop then sends it to the provider, which computes it at that period, as the CGT reserved names already do.
• One period per row. Apply the same rule to value_expression terms, filters, household_conditions and groupby_variable.
• Tell injected values apart. _inject_measure_inputs (target_materialization.py:412-416) writes provider values under the bare variable name, over the stored column in the probe copy. The adapter needs to tell an injected engine-period column from a stored one, for example by tracking the injected names or keying them by period.
• The loss-relief route. The frame route at measure_simulation.py:359-379 follows the same rule.
• Refuse a silently ignored period. At compile time, refuse a plain-column binding whose declared period the route would not honour.

2. Bind ONS road fuel at CY2024 on the 2024 columns.

• The binding. Bind the ONS CY2024 row (£37.03bn) with measurement_period: 2024 and require_matching_fact_period, read from the frame's own petrol and diesel columns. This needs item 1 to honour a declared frame period as a frame read.
• Why CY2024. It is independent of the engine. It also agrees with lcfs_consumption, which already levels petrol plus diesel to the calendar-2024 row, less the other-fuels scope share.
• Why not CY2025 through the engine. Measuring the CY2025 row through the engine would put the engine's 2024→2025 fuel projection into the weights:
  • the correction would be about +7.4% under 2.130.1 and +2.8% under 2.122.2;
  • the CY2024 binding gives about +5.8% under any engine.
• The notes. Update the binding's notes. They describe the design-weight gap as the engine's 2024→2025 uprating against ONS's own move, which is what the row was meant to measure, not what it measured.

3. Make every intended frame-period read explicit.

• The seven held rows. Item 1 would measure the seven reviewed_no_index bus and rail rows at 2025, against facts that are not uprated. For each, declare the frame period, or keep 2025 and re-word the hold.
• The unheld rows. Give the three unheld bus fare rows a verdict.

4. A certification check on the measurement period.

• The check. Re-measure every £ row, bound and diagnostic, at the engine period on the final weights through the measure provider, and compare each with its final_estimate.
• The tolerance. Flag any row outside a small tolerance, such as 0.5%, that does not declare the frame period. Whether a flag blocks or only warns is to be decided.
• The receipt. Record each row's measurement period and route (frame column or engine) in calibration_diagnostics.json, so a frame-period read shows in the receipt.

5. Tests (engine-free).

• Routing. Use a frame at time_period 2024 with a stored column, and a stub provider whose 2025 values differ. The materialized measure must equal the provider's values.
• Frame-period reads. A binding that declares measurement_period: 2024 reads the frame.
• Refusal. A declared period that the route would ignore is refused.
• Loss relief. The same three checks for the loss-relief route.

Cost.

• Code and tests. About 0.5–1 day.
• The rebuild. It rides the next national rebuild.
• The receipt. A twin receipt under experiments/ records the 76 rows and how the solve moves, on the same spine.
• Whether the local line is affected. It uses the same materialization, so its £ rows over stored columns are presumably affected too; this was not measured.

Interim correction factors

Until a file built with the fix is certified, static measures on the 810315ac2 national file (810ba980) can be scaled to undo the period error:

Base	Factor
Household road fuel (fuel duty, VAT on fuel)	×1.058 for the period error; ×1.074 also undoes the engine's extra 2024→2025 decline (label that part "engine path")
Dividends, savings interest, property income	×0.958
Private pension income	×0.977

No factor is set for:

• SMP (small);
• workplace-pension contributions (confounded by the salary-sacrifice uprating gap);
• employment income (the solve effect is untested).

Relation to #1278

• What #1278 holds. #1278 has the R1 national remeasure. Its acceptance includes the levelled road-fuel total moving less than 2% from design to final weights, and "the bound road-fuel row at design weights: R1 records it". Those items are specific to fuel.
• Why R1 depends on this. R1 run through today's materialization would read the same 2024 columns and record the fuel row as hit. R1 should measure at the engine period, or wait for item 1. Item 2 here changes R1's baseline.
• How the two split. This issue covers the general mechanism: 76 rows across six SPI families and three single rows. #1278 keeps its fuel items and links here.

Done when

• Plain-column bindings are measured at their binding's period, and a stored column is read from the frame only when the binding declares the frame period.
• ONS road fuel is bound at CY2024 on the 2024 columns.
• The ten bus and rail rows have explicit periods or re-worded holds.
• The certification check runs on every £ row and passes on a certified national build.
• A twin receipt under experiments/ records the 76 rows at the engine period and how the solve moves.

Relates #1278 (the fuel remeasure), #736 (national calibration register), #1069 and #1095 (the SPI bands and the loss-relief route), #1113 (the road-fuel row), #1123 (the uprating holds).