Move Stage 12 output planning behind country-owned typed contracts

Start by reading stage12_runtime/output_planning.py, SimulationOutputBuilder.serialize(), and the Stage 12 wire-contract definitions; the issue requires documenting current US and UK outputs and their dataset sources before implementation. Then investigate the country-owned policyengine configuration and tests to map calculated, conditional, and preserved columns. Done means a coordinated, versioned planning interface and service conversion satisfy the listed contract, validation, integration, and migration criteria.

Context

Stage 12 must determine the complete output schema before it dispatches the independent baseline and reform simulations. The current implementation in stage12_runtime/output_planning.py does this by constructing data-free Simulation objects with model_construct(), invoking country configuration functions, asking the country model to resolve entity variables, and combining those calculated variables with dataset columns known to the simulation service.

This fixed the immediate failure mode where RequestedSimulationOutput(variables=("*",)) did not materialize every column later consumed by SimulationOutputBuilder. It also gave both child simulations one typed, deterministic plan whose hash can be verified before aggregation.

Issue #707 and PR #708 add timing around this work. They intentionally do not change the planning behavior. This issue tracks the broader cleanup that should follow after #708 is integrated.

Problem

The current implementation has several fragile integration boundaries:

1. The simulation service creates partially initialized Simulation instances with Simulation.model_construct(..., dataset=None, ...). This bypasses normal model validation and assumes that every output configurator only reads policy, model, and extra_variables.
2. Output requirements are maintained in two places. The planner predicts which variables the existing report serializer will consume, but the serializer does not expose a typed declaration of those dependencies. A serializer change can therefore make the plan incomplete.
3. The simulation service contains country- and dataset-specific knowledge, including UK source columns such as constituency_code_oa and la_code_oa.
4. The current ReportOutputRequirements looks selective, but Stage 12 requires the complete aggregate profile. Only cliff-impact and labor-supply options currently change the plan.
5. Frame validation proves that entities and columns exist, but it does not express stronger contracts such as expected data types, nullability, units, or whether a column is calculated or copied from a specific dataset version.
6. The planner depends directly on mutable Simulation.extra_variables and the internal behavior of resolve_entity_variables() instead of a dedicated, supported planning interface.
7. PR #708 passes observability stages into the planner. The instrumentation is useful, but the domain planner should not need to depend on service-specific observability types.

Desired architecture

Make the policyengine package the source of truth for country-owned report-output planning, and keep the simulation service responsible for distributed execution.

policyengine package

Add a public, strongly typed planning interface that accepts explicit report requirements, baseline and reform policy configuration, country, and dataset metadata. It should return a deterministic declaration of:

• calculated variables required for each entity;
• variables added conditionally for labor-supply responses or cliff impacts;
• source-dataset columns that must be preserved;
• any schema metadata needed to validate those outputs.

The interface must not require the caller to construct an invalid or data-free Simulation. It may use a dedicated validated planning context internally.

The report serializer and output planner must share one dependency declaration. Adding a new calculation to a report must either automatically add its required variables to the plan or fail a focused contract test in the package that owns the report calculation.

Dataset-provided columns must come from typed dataset metadata or another country-owned declaration. policyengine-sim-api must not contain a hard-coded list of UK dataset columns.

The package-level types must avoid an inverse dependency on policyengine-simulation-contract. If wire-contract types remain in the simulation repository, add an explicit conversion layer between the package planning result and Stage12OutputPlan.

policyengine-sim-api

Reduce stage12_runtime/output_planning.py to orchestration:

1. Validate the Stage 12 report request.
2. Convert it into the public policyengine planning request.
3. Invoke the package planning interface once.
4. Convert the result into the versioned Stage 12 wire contract.
5. Send the identical immutable plan to the baseline and reform workers.
6. Retain deterministic sorting, hashing, bundle-provenance checks, row-identity checks, and output-frame validation.

Move timing concerns to the coordinator or inject a neutral planning-observer protocol. The package-facing planner must not import the simulation service's Stage enumeration.

Required investigation

Before implementation, document:

• every output currently consumed by SimulationOutputBuilder.serialize() for US and UK reports;
• which variables are calculated, which are source-dataset columns, and which are conditional;
• the supported dataset/version source for each preserved column;
• whether the complete Stage 12 report profile should remain the only supported profile or whether requirements should become genuinely selective;
• which validation attributes are stable enough to include in the public contract beyond entity and column names;
• the compatibility and release sequence required between policyengine bundles and policyengine-sim-api.

Acceptance criteria

• Stage 12 no longer calls Simulation.model_construct() to discover output requirements.
• policyengine exposes a documented, strongly typed report-output planning interface for both US and UK.
• The report serializer and planner use one source of truth for required calculated variables.
• Dataset-provided columns are declared by country/dataset-owned code or metadata, not hard-coded in policyengine-sim-api.
• The same deterministic plan is still sent to both Stage 12 child simulations.
• Plans remain versioned, canonically ordered, and hashable.
• Child workers still validate their materialized frames before publishing artifacts.
• The coordinator still rejects artifacts with mismatched plans, bundle provenance, roles, output schemas, or row identity.
• The planning interface fails explicitly for unsupported countries, datasets, report profiles, and unavailable source columns.
• Unit tests cover US and UK planning, budget outputs, geographic outputs, cliff impacts, active and inactive labor-supply responses, unsupported datasets, and missing declared columns.
• Contract tests prove that every column consumed by the report serializer is present in the declared output plan.
• Integration tests run baseline and reform children independently and successfully aggregate their validated artifacts.
• Observability continues to distinguish country-model loading, output configuration, variable resolution, and child-input preparation without coupling package domain types to service observability types.
• Migration and release documentation states the minimum compatible policyengine bundle and the required cross-repository merge/deployment order.

Non-goals

• Removing the typed Stage 12 plan or its hash.
• Combining the independently executed baseline and reform simulations.
• Moving distributed execution or artifact persistence into the policyengine package.
• Preserving the current internal planning implementation for backward compatibility; Stage 12 and its certified bundle can move together to the new versioned interface.

Relationship to current work

• #707 / #708 should land independently because their measurements are needed to establish the current preparation cost and confirm the effect of this cleanup.
• This work should be divided into coordinated policyengine and policyengine-sim-api changes, with the package interface released before the simulation service begins consuming it.