Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Add the simulation job store and background result collection without leases

Chiusa
  • Qualcosa non va?
#845 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

  • #848 di @anth-volk — chiusa senza merge

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
20/100
Tipo di issue
Funzionalità
Chiarezza
Specificata chiaramente
Stato di attività
Ferma
Stack tecnologico
github-actions, google-cloud, postgresql, python
Ambito
backend, cloud, databases

Direzione di ricerca

The issue specifies the simulation.jobs and simulation.job_results tables, collection and sweep routes, and concurrency tests, but names no source files or test paths. Start by locating the API's database models, route registration, and test conventions, then trace how Modal calls and Cloud Tasks are handled. Done means jobs can be submitted idempotently, results collected safely without leases, status read from the database, and the stated operator prerequisites are configured; linked PR #848 is already open.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Context

Step 1 of the API v2 migration (item 1A-4, result collection option B) needs a durable record of every calculation the Simulation Entrypoint sends to Modal, and a way to store each finished result without relying on clients that poll. The migration plan's rules: client polling never advances work, and Modal applications get no database credentials.

Scope

  • simulation.jobs: one row per calculation, recording the submitter, an input hash that includes the policyengine.py bundle version, the Modal call ID, status, timings and any error.
  • Idempotent submission: INSERT ... ON CONFLICT on the input hash, so identical requests share one job.
  • A conditional pending -> dispatching update before the Modal spawn, so one Modal call starts per job.
  • An authenticated internal collection route that Cloud Tasks calls: it checks the Modal call without waiting, writes a finished result to Postgres with a conditional update (... WHERE status = 'running') so a second collector changes nothing, and re-enqueues itself with backoff while the call runs. No Postgres leases.
  • simulation.job_results for results that are not reports (report results go to simulation.report_results, added separately).
  • A sweep route that Cloud Scheduler calls, which enqueues a collection task for unfinished jobs not checked recently.
  • Status reads that use only the database and accept either a job UUID or a Modal fc-... call ID.
  • Tests showing the conditional writes are safe when two collectors run at once.

Nothing in production submits jobs yet; no job kind is registered until Step 2.

Operator prerequisites (both environments, before merge)

  • Create the Cloud Tasks queue, its invoker service account and the IAM bindings (the Simulation Entrypoint runtime service accounts may create tasks and act as the invoker; the invoker may invoke the Cloud Run service).
  • Create the Cloud Scheduler job that calls the sweep route with the invoker's OIDC token, audience set to the stable service URL.
  • Confirm the Simulation Entrypoint's ingress setting accepts requests from Cloud Tasks and Cloud Scheduler.
  • Set the RECONCILIATION_QUEUE and RECONCILIATION_INVOKER_SERVICE_ACCOUNT variables in this repository's beta and prod GitHub environments.
Lingua principale
Python
Stelle
1
Fork
1
Merge medio
1g 21h
PR unite (30g)
57

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di PolicyEngine/policyengine-sim-api

Tutte le issue di PolicyEngine/policyengine-sim-api

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.