Define how Core's parameter caches are owned, invalidated, copied and shared

This is a design proposal, not a ready change. Start by reading how ParameterNode._get_at_instant, clear_parent_cache, add_child and get_parameters_at_instant populate and clear their caches, and how Reform.modify_parameters deep-copies the tree. Done means the maintainers have agreed on the rules in the issue, with its design sections and suggested order settled, before any cache code changes.

Why

#569, #570 and #616 tried to cut the memory and CI time of policyengine-us's YAML tests by changing how Core caches parameters. Each round of review on them found new edge cases, and each fix exposed more.

Most of the bugs found predate those PRs. Core's parameter caches have no written rules for:

• who fills them;
• what clears them;
• what a copy, a pickle or another thread sees.

Until those rules exist, there's nothing to review cache changes against. This issue proposes them. #569 and #570 stay parked as drafts until this is settled; #616 and #615 are closed.

The caches today

Cache	Filled by	Cleared by
ParameterNode._at_instant_cache: one node at instant per date	ParameterNode._get_at_instant, on node(instant)	clear_parent_cache, which Parameter.update calls and which walks up parent links. clone starts empty.
TaxBenefitSystem._parameters_at_instant_cache	get_parameters_at_instant	System construction, Reform.__init__ and the test runner after each reform. Nothing clears it after Parameter.update.
ParameterNodeAtInstant._vectorial_node	The first array index of a node at instant	Never. It lives as long as its node.
The YAML runner's _tax_benefit_system_cache	One system per combination of reforms, extensions and dotted parameter inputs	Never. It is unbounded.
Tracing wrappers	_get_at_instant caches a TracingParameterNodeAtInstant when the tree is traced	With the node cache

policyengine-us also reaches into these private attributes:

• its SPM code resets _at_instant_cache and _parameters_at_instant_cache directly, and shares one system's _parameters_at_instant_cache dict between policies;
• its per-capita uprating calls clear_parent_cache itself.

What goes wrong on master

1. Updates don't reach the system-level cache. After Parameter.update, get_parameters_at_instant returns the node it cached before the update.
2. Invalidation depends on parent links, which are often missing or wrong.
   • Scale brackets get no parent when loaded; policyengine-us has 7,777 of them. An update to a bracket's rate or threshold leaves every cache above the bracket stale.
   • Nodes built by homogenisation have no parent; policyengine-us has 9,798, such as gov.aca.lcbp.state_rating_area_cost.MP. After an update to a leaf below one of them, a fresh read of the root still returns the old value.
   • add_child sets each child as an attribute, so a child named like a node attribute replaces that attribute. policyengine-us's gov.hhs.medicaid.eligibility.categories.parent replaces the node's parent while it loads, which makes a two-node cycle:
     • Loaded on its own, an update under the node raises RecursionError.
     • Once attached, categories.parent means the tree parent again, so the child can only be reached through children.
3. Other changes clear nothing. This covers add_child, and direct edits to values_list, a scale's brackets or children.
4. Copies carry stale caches (#618, since January 2023). Reform.modify_parameters deep-copies the baseline tree together with its caches. A modifier that edits values in place then reads the baseline's values at any instant the baseline had cached.
5. Traced trees cache their wrappers. That keeps the first traced simulation's tracer alive and records later reads into it.
6. Memory.
   • Each instant materialises the whole tree: about 34,000 nodes and 16 to 33 MB on policyengine-us. The tree keeps it for the system's lifetime.
   • The YAML runner keeps a full clone for each combination, about 1.2 GB each on policyengine-us.

The attempted fixes ran into the same missing rules. Lazy nodes (#570) change their own state when read, which caused three problems:

• a missing invalidation then mixed stale and fresh values;
• copying a tree while another thread read it raised RuntimeError;
• keeping copies lazy took order-dependent copy logic and a per-thread pickle counter, which could leak.

Proposed design

1. Freeze a parameter tree once its system is built

Rule. All edits to a tree happen while a system is being built: loading, homogenisation, backdating, uprating, and a reform's apply or modify_parameters on its own clone. At the end of construction the tree is frozen. After that, Parameter.update, add_child and assignments to values_list, brackets or children raise. Code that needs a different tree clones the system or applies a reform.

What it fixes.

• Problems 1 to 4 can't happen after construction, so caches never need invalidating at run time.
• Sharing a tree between threads, or between systems as #616's path copies did, becomes safe.

Cost. Callers that update a built system must clone it first or use an explicit editing block (design 2). That needs an audit of policyengine-us, policyengine-uk, .py and the Public API for edits after construction.

2. One invalidation mechanism, independent of parent links

Rule. Edits that must stay possible bump a version number: during construction, or inside an explicit with parameters.editing(): block. Every cache entry records the version it was built at and is ignored once it is older.

How.

• The simplest form is a single module-level counter. It over-invalidates when any tree changes, but after design 1 changes are rare.
• A per-tree counter, held by one state object that every node in the tree references, is the refinement.

What it fixes. It replaces the walks up parent links, so missing parents, shadowed parents, cycles and recursion depth no longer matter.

3. One cache per concern, each with an owner

• Remove TaxBenefitSystem._parameters_at_instant_cache. get_parameters_at_instant returns the root's cached node. Measured on #569, a direct read of the tree took about 155 ns, against about 200 ns for a hit on the system cache once it was checked against the tree.
• Never cache tracing wrappers. Wrap on each read.
• _vectorial_node belongs to its node at instant and lives and dies with it.
• The YAML runner's system cache gets a bound and a documented key, as #569 proposes.
• Public methods, not private attributes. Add one to reset a system's parameter caches, and one to share caches between systems that share a frozen tree, so country packages stop editing private attributes.

4. Caches are derived data: never copied or pickled

Rule. The copy and pickle state of a ParameterNode, and of a system, leaves out its caches. A copy rebuilds them on first read.

What it fixes.

• Copies can't carry stale values (#618).
• Copies don't depend on attribute order.
• Copies stop carrying every cached tree. A warm policyengine-us tree currently takes about 8.2 s to deep-copy, and pickles to 162 MB in about 1.7 s.

Cost. #584's tests that expect a copied simulation to keep its cached vectorial node would check values instead.

5. A thread contract

A frozen tree and its caches may be read from several threads.

Rule. A cached object is immutable once it is published:

• a node at instant is fully built before it is stored;
• if built lazily, its children live in a separate map, filled under a lock, never by writing to the object's __dict__.

Result. Copying stays safe while other threads read.

6. Children are not attributes

Rule. Stop setattr(node, name, child). ParameterNode.__getattr__ looks a name up in children only after the node's own attributes. So node.parent, node.name and node.metadata always mean the node's own. A child with such a name is reached through node.children[...] or get_child.

Load-time check. Warn about, or reject, child names that collide with node attributes.

7. One way to attach a child

Rule. Every attachment goes through add_child, or an equivalent add_bracket for scales, which sets parent, and the tree-state reference if design 2 is per tree. That covers:

• loading;
• scale brackets;
• homogenisation;
• clone;
• reforms that append brackets;
• path copies.

Check. A test helper checks the invariant that every child's parent is the node holding it, on Core's country template and in the country packages' test suites.

Suggested order

1. Designs 6 and 7. They're small, fix real bugs on master (cycles, nodes without parents, bracket parents) and don't change semantics.
2. Designs 3 and 4. Remove the system-level cache and stop copying caches. This fixes #618.
3. Designs 1 and 2. Freeze trees after construction, after auditing the callers that update built systems.
4. Design 5, then the memory work. Revisit lazy nodes (#570) and the runner changes on top of frozen trees: #569, and #616's per-case path copies and forked runner. With immutable trees, laziness needs no invalidation at all.

Related

• #569 and #570: draft PRs, parked. Most of their review findings trace back to the missing rules above.
• #615 and #616: closed. The CI runner changes will be revisited after this.
• #618: open. Designs 3 and 4 resolve it.