Hacktoberfest 2026 : les issues que les mainteneurs ont marquées pour octobre, ouvertes et accessibles aux débutants. Parcourir les issues Hacktoberfest

Add the simulation job store and background result collection without leases

Fermée
  • Quelque chose ne va pas ?
#845 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 1 jour

Personne n'a encore pris cette issue.

  • #848 par @anth-volk — fermée sans fusion

Évaluation

Difficulté
5/5
Temps estimé
Plus d'une semaine
Accessibilité débutants
20/100
Type d'issue
Fonctionnalité
Clarté
Clairement spécifiée
Activité
À l'abandon
Stack technique
github-actions, google-cloud, postgresql, python
Domaine
backend, cloud, databases

Piste de recherche

The issue specifies the simulation.jobs and simulation.job_results tables, collection and sweep routes, and concurrency tests, but names no source files or test paths. Start by locating the API's database models, route registration, and test conventions, then trace how Modal calls and Cloud Tasks are handled. Done means jobs can be submitted idempotently, results collected safely without leases, status read from the database, and the stated operator prerequisites are configured; linked PR #848 is already open.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

Context

Step 1 of the API v2 migration (item 1A-4, result collection option B) needs a durable record of every calculation the Simulation Entrypoint sends to Modal, and a way to store each finished result without relying on clients that poll. The migration plan's rules: client polling never advances work, and Modal applications get no database credentials.

Scope

  • simulation.jobs: one row per calculation, recording the submitter, an input hash that includes the policyengine.py bundle version, the Modal call ID, status, timings and any error.
  • Idempotent submission: INSERT ... ON CONFLICT on the input hash, so identical requests share one job.
  • A conditional pending -> dispatching update before the Modal spawn, so one Modal call starts per job.
  • An authenticated internal collection route that Cloud Tasks calls: it checks the Modal call without waiting, writes a finished result to Postgres with a conditional update (... WHERE status = 'running') so a second collector changes nothing, and re-enqueues itself with backoff while the call runs. No Postgres leases.
  • simulation.job_results for results that are not reports (report results go to simulation.report_results, added separately).
  • A sweep route that Cloud Scheduler calls, which enqueues a collection task for unfinished jobs not checked recently.
  • Status reads that use only the database and accept either a job UUID or a Modal fc-... call ID.
  • Tests showing the conditional writes are safe when two collectors run at once.

Nothing in production submits jobs yet; no job kind is registered until Step 2.

Operator prerequisites (both environments, before merge)

  • Create the Cloud Tasks queue, its invoker service account and the IAM bindings (the Simulation Entrypoint runtime service accounts may create tasks and act as the invoker; the invoker may invoke the Cloud Run service).
  • Create the Cloud Scheduler job that calls the sweep route with the invoker's OIDC token, audience set to the stable service URL.
  • Confirm the Simulation Entrypoint's ingress setting accepts requests from Cloud Tasks and Cloud Scheduler.
  • Set the RECONCILIATION_QUEUE and RECONCILIATION_INVOKER_SERVICE_ACCOUNT variables in this repository's beta and prod GitHub environments.
Langage dominant
Python
Étoiles
1
Forks
1
Merge moyen
1 j 16 h
PR mergées (30 j)
61

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de PolicyEngine/policyengine-sim-api

Toutes les issues de PolicyEngine/policyengine-sim-api

Issues similaires

Plus d'issues Python

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.