build_from_populations leaves each holder bound to the simulation that made it

Aperta
  • Qualcosa non va?
#636 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 2 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
45/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
python
Ambito
backend, databases

Direzione di ricerca

Start by reading Simulation.build_from_populations, Holder._set, Holder._is_input, and the holder handling in Simulation.clone and branch creation; the issue gives a core-only reproduction for each case. Trace how holders and input records are rebound during a rebuild, then run the reproductions to check that the receiver, its clone, and its branch all retain and export supplied inputs correctly.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Simulation.build_from_populations(populations) points each population at the new simulation (link_to_entities_instances), but each holder keeps holder.simulation, the simulation that made it. The holder uses that simulation to decide whether a store is an input and to record it (Holder._set reads its _user_input_contexts and adds to its _user_input_keys), to answer Holder._is_input, and in delete_arrays and fast-cache eviction. So after a rebuild from another simulation's populations, an input set on the new simulation goes into the other simulation's record. The new simulation and its clones then neither export it nor count it as an input.

Reproduction (core only, 3.33.12; build_from_populations reads the same on master at 04cda03b7, 3.33.13)

from policyengine_core.country_template import CountryTaxBenefitSystem
from policyengine_core.simulations import SimulationBuilder

tbs = CountryTaxBenefitSystem()
households = {"h": {"parents": ["a"]}}
def build(salary):
    return SimulationBuilder().build_from_entities(
        tbs, {"persons": {"a": {"salary": salary}}, "households": households}
    )

donor = build({"2018-01": 1000})
receiver = build({"2018-01": 500})
receiver.build_from_populations(donor.populations)
holder = receiver.populations["person"].get_holder("salary")
print("population bound to receiver:", receiver.populations["person"].simulation is receiver)
print("holder bound to receiver:", holder.simulation is receiver, "| to donor:", holder.simulation is donor)
receiver.set_input("salary", "2018-02", [700])
key = ("salary", "default", __import__("policyengine_core").periods.period("2018-02"))
print("receiver records the input set on it:", key in receiver._user_input_keys)
print("donor records it instead:", key in donor._user_input_keys)
print("receiver exports:", sorted(c for c in receiver.to_input_dataframe().columns if c.startswith("salary")))
clone = receiver.clone()
print("clone exports:", sorted(c for c in clone.to_input_dataframe().columns if c.startswith("salary")))

Output:

population bound to receiver: True
holder bound to receiver: False | to donor: True
receiver records the input set on it: False
donor records it instead: True
receiver exports: ['salary__2018-01']
clone exports: ['salary__2018-01']

The receiver still exports salary__2018-01 because its own record of the input it held before the rebuild (#605) now reads the donor's holder, which stores 1,000 there.

A clone or branch of the receiver then loses the donor's inputs too. Simulation.clone copies the receiver's _user_input_keys, and Holder.clone binds each copied holder to the clone, so the copy reads a record that never held the donor's keys:

"""Core only: a clone of a rebuilt simulation loses the donor's inputs."""
from policyengine_core.country_template import CountryTaxBenefitSystem
from policyengine_core.periods import period
from policyengine_core.simulations import SimulationBuilder

tbs = CountryTaxBenefitSystem()
households = {"h": {"parents": ["a"]}}


def build(persons):
    return SimulationBuilder().build_from_entities(
        tbs, {"persons": {"a": persons}, "households": households}
    )


donor = build({"salary": {"2018-01": 1000}})
receiver = build({})
receiver.build_from_populations(donor.populations)
january = period("2018-01")
for label, simulation in (("receiver", receiver), ("clone", receiver.clone()), ("branch", receiver.get_branch("b"))):
    holder = simulation.populations["person"].get_holder("salary")
    print(
        label,
        "| stores:", holder.get_array(january, simulation.branch_name),
        "| counts it as an input:", holder._is_input(january, simulation.branch_name),
        "| exports:", sorted(c for c in simulation.to_input_dataframe().columns if c.startswith("salary")),
    )

receiver | stores: [1000.] | counts it as an input: True | exports: []
clone | stores: [1000.] | counts it as an input: False | exports: []
branch | stores: [1000.] | counts it as an input: False | exports: []

The receiver counts the value as an input only because its holder still reads the donor's record, and it does not export it; the clone and the branch treat it as calculated, so code that defers to a supplied value (policyengine-uk's housing benefit income override does) recalculates it.

Related

  • #605: the rebuild keeps the record of the replaced holders' inputs.
  • PolicyEngine/policyengine-uk#2234 works around these in its Simulation.build_from_populations (adopt_population_records in policyengine_uk/utils/supplied_inputs.py): it binds each holder from another simulation to the new one, carries that simulation's input records for the values the holder still stores, drops the values it calculated, and forgets the records of values the rebuild removed.

A review of that PR found the clone and branch case.

Lingua principale
Python
Stelle
22
Fork
30
Merge medio
4g 20h
PR unite (30g)
26

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di PolicyEngine/policyengine-core

Tutte le issue di PolicyEngine/policyengine-core

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.