Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Register weighted-microdata rules: weights aren't values

Abierto
  • ¿Algo no está bien?
#806 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

@anth-volk ya está trabajando en esto.

Desde el 14/10/2026.

  • #807 de @anth-volk — abierto

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
30/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
pandas, python
Área
data, testing

Línea de trabajo

Start with rules/registry.toml to see how existing rules and their AST guards and proofs are declared, then read simulation_output_common.py, where raw values are read from weighted frames. The output-builder tests that currently use plain pd.DataFrames are the other place to look. Done means the four named rules are registered with guards and proofs, the confined-read exception list is in place, and the builder tests use one shared weighted factory.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Problem

The executor's entity frames are microdf MicroDataFrames weighted by <entity>_weight, so .sum(), .mean() and .count() on any column are already weighted.

In #766 (fixed by #767), build_budgetary_impact passed "household_weight" to _sum_output_variable, the helper it uses for ordinary values. That weighted the weight column by itself, giving the sum of squared weights.

  • Effect: budget.households was about 10^4 times too high on every run (New York 2026: 135.6 billion, against about 7.5 million households). Clients divide by it, so the average change per household was about 10^4 times too small.
  • Why it escaped: the builder test used plain pd.DataFrames, which carry no weights.

Nothing stops the same class of mistake from coming back.

Proposal

Phase 1 step 1.9. Register the rule that weights aren't values in rules/registry.toml, with AST guards and proofs:

  • sim.microdata.weights-not-values (doc-only): weights, values, labels and counts each allow different operations.
  • sim.microdata.weights-not-aggregated-as-values: a weight is never the variable of a weighted aggregator, and never frame[<weight>] followed by a weighted reduction.
  • sim.microdata.weight-stripping-confined: raw values are read from a weighted frame only in simulation_output_common.py.
    • Counting entities is always allowed there.
    • Any other read there is a recorded exception, in a list that may only shrink.
  • sim.microdata.builder-tests-use-weighted-frames: output-builder tests build entity frames through one shared weighted factory, never as plain pd.DataFrames.

Alongside, two cheap runtime checks that the AST guards can't replace:

  • change totals (reform minus baseline) log a warning when the two sides carry different weights;
  • an implausible published household count fails the run.

Decisions

  • UK local-authority handoff. The breakdown hands policyengine's longwise geography impact a plain household frame, and that function weights by household_weight itself. The handoff moves into simulation_output_common.py as the stripping rule's one recorded exception. Removing it is tracked separately.
  • Weight mismatch: warn first. It logs a warning and still publishes the total. It becomes an error once logs show that no legitimate run produces a mismatch.
  • Household count: fail the run. A failed run is better than a headline that's 10^4 times wrong.
Lenguaje dominante
Python
Estrellas
1
Forks
1
Merge medio
1 d 11 h
PR fusionados (30 d)
50

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de PolicyEngine/policyengine-sim-api

Todos los issues de PolicyEngine/policyengine-sim-api

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.