Hacktoberfest 2026 : les issues que les mainteneurs ont marquées pour octobre, ouvertes et accessibles aux débutants. Parcourir les issues Hacktoberfest

Register weighted-microdata rules: weights aren't values

Ouverte
  • Quelque chose ne va pas ?
#806 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 1 jour

@anth-volk y travaille déjà.

Depuis le 14/10/2026.

  • #807 par @anth-volk — ouverte

Évaluation

Difficulté
4/5
Temps estimé
3-5 jours
Accessibilité débutants
30/100
Type d'issue
Fonctionnalité
Clarté
Plutôt claire
Activité
Active
Stack technique
pandas, python
Domaine
data, testing

Piste de recherche

Start with rules/registry.toml to see how existing rules and their AST guards and proofs are declared, then read simulation_output_common.py, where raw values are read from weighted frames. The output-builder tests that currently use plain pd.DataFrames are the other place to look. Done means the four named rules are registered with guards and proofs, the confined-read exception list is in place, and the builder tests use one shared weighted factory.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

Problem

The executor's entity frames are microdf MicroDataFrames weighted by <entity>_weight, so .sum(), .mean() and .count() on any column are already weighted.

In #766 (fixed by #767), build_budgetary_impact passed "household_weight" to _sum_output_variable, the helper it uses for ordinary values. That weighted the weight column by itself, giving the sum of squared weights.

  • Effect: budget.households was about 10^4 times too high on every run (New York 2026: 135.6 billion, against about 7.5 million households). Clients divide by it, so the average change per household was about 10^4 times too small.
  • Why it escaped: the builder test used plain pd.DataFrames, which carry no weights.

Nothing stops the same class of mistake from coming back.

Proposal

Phase 1 step 1.9. Register the rule that weights aren't values in rules/registry.toml, with AST guards and proofs:

  • sim.microdata.weights-not-values (doc-only): weights, values, labels and counts each allow different operations.
  • sim.microdata.weights-not-aggregated-as-values: a weight is never the variable of a weighted aggregator, and never frame[<weight>] followed by a weighted reduction.
  • sim.microdata.weight-stripping-confined: raw values are read from a weighted frame only in simulation_output_common.py.
    • Counting entities is always allowed there.
    • Any other read there is a recorded exception, in a list that may only shrink.
  • sim.microdata.builder-tests-use-weighted-frames: output-builder tests build entity frames through one shared weighted factory, never as plain pd.DataFrames.

Alongside, two cheap runtime checks that the AST guards can't replace:

  • change totals (reform minus baseline) log a warning when the two sides carry different weights;
  • an implausible published household count fails the run.

Decisions

  • UK local-authority handoff. The breakdown hands policyengine's longwise geography impact a plain household frame, and that function weights by household_weight itself. The handoff moves into simulation_output_common.py as the stripping rule's one recorded exception. Removing it is tracked separately.
  • Weight mismatch: warn first. It logs a warning and still publishes the total. It becomes an error once logs show that no legitimate run produces a mismatch.
  • Household count: fail the run. A failed run is better than a headline that's 10^4 times wrong.
Langage dominant
Python
Étoiles
1
Forks
1
Merge moyen
1 j 7 h
PR mergées (30 j)
40

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de PolicyEngine/policyengine-sim-api

Toutes les issues de PolicyEngine/policyengine-sim-api

Issues similaires

Plus d'issues Python

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.