API v2: report validations as an append-only, provenance-keyed resource on reports

@anth-volk — raising this now so the v2 reports stage can be designed with it in mind, rather than retrofitting it.

Context

policyengine-app-v2 now validates every flagship report on two data sources: the calibration dashboard (how well the microdata fits administrative targets for the variables a reform moves) and the scorecard (PolicyEngine versus external measurement for the programs it touches). The variables come from a dependency map traced from the model (PolicyEngine/policyengine-core#543); the matching is in PolicyEngine/policyengine-app-v2#1180 and #1184. On first view the app pins what it matched against — populace release, model version, per-variable fit, scorecard programs — and later views report drift against that pin (#1185).

Today the pin lives in the browser's localStorage, because the app's own reports table was never connected to a database, deliberately, since v2 is where user-owned report data is going. The validation snapshot is a fact about the base report, not about one user's view of it, so it should live with the report in v2 rather than in the app or on a user association.

Proposal: report_validations

One row per validation run, never overwritten, keyed to the base report:

Endpoints:

• GET /v2/reports/{id}/validations → latest plus history, newest first
• POST /v2/reports/{id}/validations → append a run (write-once per run; never update)

Drift is then a diff between the latest row's populace_release_id / model_version and the current ones, which is how the tracker already treats bill validations.

JSON the app writes today

This is the shape already in production behind the app's localStorage fallback; calibration and scorecard map onto it directly, and the top-level keys become columns:

{
  "matched_at": "2026-09-07T08:55:20.772Z",
  "map_model_version": "1.808.0",
  "calibration": {
    "release_id": "populace-us-2024-buildp-sparse-rmloss100-cae8640-20260728T011454Z",
    "geography": "US",
    "reached_count": 66,
    "matches": [
      {
        "variable": "refundable_ctc",
        "depth": 2,
        "ring": "mechanism",
        "target_count": 2,
        "mean_abs_relative_error": 0.008,
        "worst": { "name": "irs_soi.ty2022.historic_table_2.us.all.actc_amount@2024", "source": "irs_soi", "geography": "US", "relative_error": -0.008 }
      }
    ]
  },
  "scorecard": { "programs": ["ctc_refund"] }
}

ring is one of primary (0–1 hops from the parameter), mechanism (2–4), downstream (5+). Errors are fractions.

Staged path

1. Table and endpoints land with the reports stage. Contained addition; the app changes only the store it posts to.
2. The app posts its snapshot on the first settled view instead of writing localStorage. Durable and shared across viewers from that day.
3. Computation moves into the API worker at report completion, once core#543 releases and the dependency map ships as a release asset per model version. The worker knows the exact model version it just used, so provenance stops being inferred.
4. A scheduled job re-validates reports whose pinned release or model is behind, appending rows. Drift becomes something the system records rather than something a browser notices.

Steps 1–2 need nothing from core; 3–4 wait on core#543.

Questions for you

• Does a separate table fit the reports stage, or would you rather hang it off the base report resource? Separate keeps history and avoids widening reports.
• Should POST be open to the app (anonymous trust model, like today's association writes) or restricted to the worker once step 3 lands?
• Anything in the v2 conventions (naming, envelope, pagination) the endpoint shapes above should follow?

References: PolicyEngine/policyengine-app-v2#1180, #1184, #1185, #1186; PolicyEngine/policyengine-core#543.