Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Scoped UK runs give the region UK-wide business rates and region-relative ranks

Open
  • Is something wrong?
#567 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
15/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
python
Domain
data

Research direction

The problem sits in how policyengine-uk formulas (shareholding, business_rates, household_income_decile, in_relative_poverty_bhc) normalise over the dataset, so scoped runs in policyengine.py 6.2.3 see region totals. Start from the row-filter and region handling added in #565, and compare a scoped run against the national run for the same households. Done means a maintainer has chosen between the two options in the issue, and the chosen approach is implemented and checked against the national-run figures in the table.

Written by the indexing model from the issue text.

Description

What happens

A UK simulation scoped by row filter (nation, constituency or local-authority regions, or any RowFilterStrategy/RegionGroupStrategy) runs policyengine-uk on the filtered households only. Several policyengine-uk formulas normalise over the households in the dataset. On a filtered dataset they normalise over the region instead of the UK, so the region's households receive UK-wide totals and region-relative ranks.

Measured on 2026-10-08/09 with policyengine-uk 2.102.3 on the certified enhanced_frs_2024_25_year_2026.h5 (baseline policy, 2026). The nation rows were re-run on main at 07bae75 (6.2.3, #565's region filters) and match the first measurement exactly. Each figure compares a scoped run with the same households in the national run:

scope households business rates, scoped same households, national run household net income, scoped same households, national run
country/england 37,961 £31.73bn £27.28bn £1,487.19bn £1,492.28bn
country/scotland 5,405 £31.73bn £2.37bn £103.98bn £139.15bn
country/wales 3,821 £31.73bn £1.83bn £49.28bn £85.06bn
country/northern_ireland 5,659 £31.73bn £0.25bn £6.07bn £43.75bn
constituency W07000105 143 £31.73bn £0.05bn −£34.45bn £3.48bn

£31.73bn is the UK business-rates total. The national run splits it £27.28bn / £2.37bn / £1.83bn / £0.25bn across the four nations; every scoped run puts all of it on the region.

Which variables (Scotland, 5,405 households)

variable records that differ scoped total national run, same records
shareholding 484 1.00 0.07
business_rates 4,375 £31.73bn £2.37bn
corporate_sdlt 4,375 £3.13bn £0.23bn
expected_sdlt 4,375 £3.13bn £0.23bn
corporate_land_value 4,375 £2,058.6bn £153.4bn
household_income_decile 1,340
household_wealth_decile 2,072
pre_budget_change_ons_household_income_decile 3,955
in_relative_poverty_bhc / _ahc 45 / 89 736k / 751k weighted 651k / 709k
attends_private_school (persons) 23

State Pension, LHA rates, weights and the absolute poverty thresholds match. The reform-only incidence variables (corporate_incident_tax_revenue_change, consumer_incident_tax_revenue_change, employer_ni_response_*) are zero in the baseline, so this run does not test them.

Mechanism (policyengine-uk 2.102.3, read this session)

  • shareholding = household corporate_sector_wealth / Σ(corporate_sector_wealth × household_weight) over the dataset. business_rates = shareholding × UK business-rates revenue (all four nations). corporate_sdlt = shareholding × UK corporate stamp-duty revenue. corporate_land_value divides by the same dataset total. On a filtered dataset the shares sum to 1 over the region.
  • household_income_decile ranks equiv_hbai_household_net_income within the dataset, so scoped runs report within-region deciles.
  • in_relative_poverty_bhc compares income with 60% of the dataset's person-weighted median, so scoped runs use the region's median, not the UK's.

Who is affected

  • policyengine.py 6.2.3: since #565, nation-scoped runs no longer raise; they return the totals above. Constituency and local-authority regions built with include_constituencies / include_local_authorities produce them too.
  • The simulation API: app-v2 → policyengine-api → simulation gateway → executor resolves UK nation regions through this registry. The executor still pins 6.2.1, where nation runs raise. PolicyEngine/policyengine-sim-api#733 (draft) moves it to 6.2.3, and PolicyEngine/policyengine-api#3877 restores the Scotland population test on it. Once those deploy, API nation results carry these totals unless this is fixed first.

Options

  1. Run nationally, then filter outputs, for UK row-filter strategies. Scoped results equal the national run restricted to the region by construction, including under reforms. A UK national run takes seconds. Regional deciles become UK deciles, unlike US state runs, which keep filtering before the run.
  2. Pass national values into the scoped run (for example shareholding from the unscoped baseline). This keeps within-region deciles. It cannot cover reform-dependent quantities such as the relative poverty median under a reform, or the reform-only incidence variables.

Option 1 changes what regional decile outputs mean, so it needs a decision.

Dominant language
Python
Stars
8
Forks
10
Avg merge
18h 24m
Merged PRs (30d)
18

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from PolicyEngine/policyengine.py

All issues in PolicyEngine/policyengine.py

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.