Hacktoberfest 2026: die Issues, die Maintainer für den Oktober markiert haben – offen und einsteigerfreundlich. Hacktoberfest-Issues durchsuchen

Add the simulation job store and background result collection without leases

Geschlossen
  • Stimmt etwas nicht?
#845 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

  • #848 von @anth-volk — ohne Merge geschlossen

Bewertung

Schwierigkeit
5/5
Geschätzter Aufwand
Über eine Woche
Anfängerfreundlichkeit
20/100
Issue-Typ
Feature
Klarheit
Klar beschrieben
Aktivitätsstatus
Veraltet
Tech-Stack
github-actions, google-cloud, postgresql, python
Bereich
backend, cloud, databases

Rechercherichtung

The issue specifies the simulation.jobs and simulation.job_results tables, collection and sweep routes, and concurrency tests, but names no source files or test paths. Start by locating the API's database models, route registration, and test conventions, then trace how Modal calls and Cloud Tasks are handled. Done means jobs can be submitted idempotently, results collected safely without leases, status read from the database, and the stated operator prerequisites are configured; linked PR #848 is already open.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

Context

Step 1 of the API v2 migration (item 1A-4, result collection option B) needs a durable record of every calculation the Simulation Entrypoint sends to Modal, and a way to store each finished result without relying on clients that poll. The migration plan's rules: client polling never advances work, and Modal applications get no database credentials.

Scope

  • simulation.jobs: one row per calculation, recording the submitter, an input hash that includes the policyengine.py bundle version, the Modal call ID, status, timings and any error.
  • Idempotent submission: INSERT ... ON CONFLICT on the input hash, so identical requests share one job.
  • A conditional pending -> dispatching update before the Modal spawn, so one Modal call starts per job.
  • An authenticated internal collection route that Cloud Tasks calls: it checks the Modal call without waiting, writes a finished result to Postgres with a conditional update (... WHERE status = 'running') so a second collector changes nothing, and re-enqueues itself with backoff while the call runs. No Postgres leases.
  • simulation.job_results for results that are not reports (report results go to simulation.report_results, added separately).
  • A sweep route that Cloud Scheduler calls, which enqueues a collection task for unfinished jobs not checked recently.
  • Status reads that use only the database and accept either a job UUID or a Modal fc-... call ID.
  • Tests showing the conditional writes are safe when two collectors run at once.

Nothing in production submits jobs yet; no job kind is registered until Step 2.

Operator prerequisites (both environments, before merge)

  • Create the Cloud Tasks queue, its invoker service account and the IAM bindings (the Simulation Entrypoint runtime service accounts may create tasks and act as the invoker; the invoker may invoke the Cloud Run service).
  • Create the Cloud Scheduler job that calls the sweep route with the invoker's OIDC token, audience set to the stable service URL.
  • Confirm the Simulation Entrypoint's ingress setting accepts requests from Cloud Tasks and Cloud Scheduler.
  • Set the RECONCILIATION_QUEUE and RECONCILIATION_INVOKER_SERVICE_ACCOUNT variables in this repository's beta and prod GitHub environments.
Vorherrschende Sprache
Python
Sterne
1
Forks
1
Ø Merge
1 T. 16 Std.
Gemergte PRs (30 T.)
61

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus PolicyEngine/policyengine-sim-api

Alle Issues in PolicyEngine/policyengine-sim-api

Ähnliche Issues

Weitere Issues zu Python

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.