Segmented US national runs wait on stuck regional children until the parent times out
I maintainer di solito rispondono entro 1 giorno
Valutazione
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Idoneità per principianti
- 25/100
- Tipo di issue
- Bug
- Chiarezza
- Abbastanza chiara
- Stato di attività
- Attiva
- Stack tecnologico
- python
- Ambito
- backend, distributed-systems
Direzione di ricerca
Start with the parent run_simulation function, which waits on regional child runs with no per-child deadline and is killed by the 3,600 s Modal timeout. The payload does not name the files or tests involved, so locate the orchestration code yourself. Done means a stuck child is cancelled and retried once, the run fails with the region group named, and the parent cancels its children before its own timeout. PR #795 already targets this issue, so check it before starting.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Problem
A segmented US national run waits on each regional child with no deadline of its own. If a child never finishes, the parent run_simulation waits until its 3,600 s Modal timeout kills it. Because the parent is killed rather than failing in code, nothing cancels its children, and they keep running.
On 9 October 2026, both US national reports for 2023 hit this. They used the same policy and started at about 05:45 UTC.
- Their 20 regional children were re-run 180 and 131 times on new containers, and none finished. The few that logged an error were interrupted during dataset loading after 229–517 s.
- First report: the parent ended at 3,599.8 s with Modal's
InputCancellation("Input was cancelled by user"), which is its timeout. Its children kept restarting for about another 50 minutes. - Second report: the parent failed after 3,167 s with
Server has lost track of input, from polling a restarting child.
Reports for other years needed at most three extra child runs, and all succeeded.
Expected
- A child with no result within a reasonable deadline is cancelled and retried once.
- The run fails with a clear message naming the region group if the retry also fails or runs out of time.
- The parent fails in code, and cancels its children, before its Modal timeout.
Follow-up (not in scope)
Find why the 2023 children's containers kept being terminated, which needs Modal container logs. Respawned children also start new traces with no parent span.
- Lingua principale
- Python
- Stelle
- 1
- Fork
- 1
- Merge medio
- 1g 4h
- PR unite (30g)
- 27
Preparare l'ambiente
- Nessun Dockerfile né file Docker Compose
- Ha un modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di PolicyEngine/policyengine-sim-api
-
Integration tests fail due to missing UV virtual environment setupForse già presa Una pull request collegata a questa issue è aperta o già unita. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 65/100
PolicyEngine/policyengine-sim-api#314 ·
I maintainer di solito rispondono entro 1 giorno
-
Update policyengine to 6.3.4Forse già presa @policyengine l’ha presa oggi. Aperta
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 40/100
PolicyEngine/policyengine-sim-api#796 ·
I maintainer di solito rispondono entro 1 giorno
-
Stage 12 coordinator can outlive its timeout while waiting and leaves workers runningForse già presa @anth-volk l’ha presa oggi. Aperta
Difficoltà 3/5 1-2 giorni Idoneità per principianti 20/100
PolicyEngine/policyengine-sim-api#792 ·
I maintainer di solito rispondono entro 1 giorno
-
Stage 12 failure records keep only the exception class nameForse già presa @anth-volk l’ha presa oggi. Aperta
Difficoltà 3/5 1-2 giorni Idoneità per principianti 25/100
PolicyEngine/policyengine-sim-api#790 ·
I maintainer di solito rispondono entro 1 giorno
-
Stage 12 runs exhaust the database session pooler's 15-client limitForse già presa @anth-volk l’ha presa oggi. Aperta
Difficoltà 3/5 1-2 giorni Idoneità per principianti 25/100
PolicyEngine/policyengine-sim-api#788 ·
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di PolicyEngine/policyengine-sim-api
Issue simili
-
[Bug]: Streamed /v1/chat/completions sends unredacted agent error text (keys) in the final chunkForse già presa @rodricksz4h5 l’ha presa oggi. Apertaarea/streaming comp/gateway P2 type/security
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
NousResearch/hermes-agent#138245 ·
I maintainer di solito rispondono entro 1 giorno
-
[oblt-aw][autodoc] Update user onboarding guide for mandatory trigger CODEOWNERS stepForse già presa @github-actions l’ha presa oggi. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 66/100
I maintainer di solito rispondono entro 1 giorno
-
correction metadata
Difficoltà 2/5 Meno di un'ora Idoneità per principianti 68/100
acl-org/acl-anthology#10378 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
cloudtrail_multi_region_enabled_logging_management_events fails when the advanced event selector has a custom nameForse già presa @12YYCCYY16 l’ha presa oggi. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
prowler-cloud/prowler#13033 ·
I maintainer di solito rispondono entro 1 giorno
-
enhancement
Difficoltà 1/5 1-3 ore Idoneità per principianti 88/100
graygarbage/HermesKey#23 ·