Hacktoberfest 2026: những issue maintainer đã đánh dấu cho tháng Mười, đang mở và phù hợp người mới. Xem issue Hacktoberfest

API v2: report validations as an append-only, provenance-keyed resource on reports

Đang mở
  • Có gì không ổn?
#3,818 0 bình luận 0 reaction 1 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

@anth-volk đang làm issue này rồi.

Từ ngày 8/9/2026.

Đánh giá

Issue này chưa được đánh giá.

Mô tả

@anth-volk — raising this now so the v2 reports stage can be designed with it in mind, rather than retrofitting it.

Context

policyengine-app-v2 now validates every flagship report on two data sources: the calibration dashboard (how well the microdata fits administrative targets for the variables a reform moves) and the scorecard (PolicyEngine versus external measurement for the programs it touches). The variables come from a dependency map traced from the model (PolicyEngine/policyengine-core#543); the matching is in PolicyEngine/policyengine-app-v2#1180 and #1184. On first view the app pins what it matched against — populace release, model version, per-variable fit, scorecard programs — and later views report drift against that pin (#1185).

Today the pin lives in the browser's localStorage, because the app's own reports table was never connected to a database, deliberately, since v2 is where user-owned report data is going. The validation snapshot is a fact about the base report, not about one user's view of it, so it should live with the report in v2 rather than in the app or on a user association.

Proposal: report_validations

One row per validation run, never overwritten, keyed to the base report:

column type purpose
id uuid
report_id fk → reports, indexed the base API report
computed_at timestamptz when the run happened
model_version text policyengine-us the report was computed with
dependency_map_fingerprint text which traced map resolved paths to variables (core#543 stamps this)
populace_release_id text, nullable which calibration release was read
scorecard_build text, nullable which scorecard file was read
calibration jsonb, nullable geography, reached count, per-variable ring/depth/fit/worst target
scorecard jsonb, nullable programs (and later rows)

Endpoints:

  • GET /v2/reports/{id}/validations → latest plus history, newest first
  • POST /v2/reports/{id}/validations → append a run (write-once per run; never update)

Drift is then a diff between the latest row's populace_release_id / model_version and the current ones, which is how the tracker already treats bill validations.

JSON the app writes today

This is the shape already in production behind the app's localStorage fallback; calibration and scorecard map onto it directly, and the top-level keys become columns:

{
  "matched_at": "2026-09-07T08:55:20.772Z",
  "map_model_version": "1.808.0",
  "calibration": {
    "release_id": "populace-us-2024-buildp-sparse-rmloss100-cae8640-20260728T011454Z",
    "geography": "US",
    "reached_count": 66,
    "matches": [
      {
        "variable": "refundable_ctc",
        "depth": 2,
        "ring": "mechanism",
        "target_count": 2,
        "mean_abs_relative_error": 0.008,
        "worst": { "name": "irs_soi.ty2022.historic_table_2.us.all.actc_amount@2024", "source": "irs_soi", "geography": "US", "relative_error": -0.008 }
      }
    ]
  },
  "scorecard": { "programs": ["ctc_refund"] }
}

ring is one of primary (0–1 hops from the parameter), mechanism (2–4), downstream (5+). Errors are fractions.

Staged path

  1. Table and endpoints land with the reports stage. Contained addition; the app changes only the store it posts to.
  2. The app posts its snapshot on the first settled view instead of writing localStorage. Durable and shared across viewers from that day.
  3. Computation moves into the API worker at report completion, once core#543 releases and the dependency map ships as a release asset per model version. The worker knows the exact model version it just used, so provenance stops being inferred.
  4. A scheduled job re-validates reports whose pinned release or model is behind, appending rows. Drift becomes something the system records rather than something a browser notices.

Steps 1–2 need nothing from core; 3–4 wait on core#543.

Questions for you

  • Does a separate table fit the reports stage, or would you rather hang it off the base report resource? Separate keeps history and avoids widening reports.
  • Should POST be open to the app (anonymous trust model, like today's association writes) or restricted to the worker once step 3 lands?
  • Anything in the v2 conventions (naming, envelope, pagination) the endpoint shapes above should follow?

References: PolicyEngine/policyengine-app-v2#1180, #1184, #1185, #1186; PolicyEngine/policyengine-core#543.

Ngôn ngữ chính
Python
Star
18
Fork
33
Merge trung bình
1 ngày 3 giờ
Pull request đã merge (30 ngày)
23

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của PolicyEngine/policyengine-api

Tất cả issue của PolicyEngine/policyengine-api

Issue tương tự

Thêm issue về Python

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.