A failed fetch leaves the acquired batch to the reaper
Los mantenedores suelen responder en 1 día
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 5/5
- Tiempo estimado
- Más de una semana
- Aptitud para principiantes
- 35/100
- Tipo de issue
- Error
- Claridad
- Bastante claro
- Estado de actividad
- Activo
- Área
- backend, databases, distributed-systems
Línea de trabajo
Start with acquire.lua and trace how leased tasks are handled when TaskResult deserialization fails. Then inspect maintain_worker_pool and the reaper's lease-expiry behavior. Done means fetch failures no longer silently lose leased tasks and the intended worker-drain behavior remains intact.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Deferred from the task-prefetching review (crew run): the error path is now loud and the worker exits non-zero, but returning the tasks is the larger design piece.
What
acquire.lua leases every task of a batch (ZADD into the running set with the lease deadline, task data rewritten RUNNING) before the reply is deserialized. If the fetcher then raises — a poison payload that fails TaskResult reconstruction, a renamed or removed task function, a broker error mid-reply — the whole leased batch is discarded:
- the batch is never buffered, never executed, and never acknowledged
- the reaper moves it to the failed history after
lease_ttl(default 1 hour) and never back to READY, so the tasks are lost unless an operator requeues them - with
--exit-empty, dead workers arecontinued bymaintain_worker_pool, so a broker outage across the fetchers can end the whole drain run as if the queue were empty
Reproduced dynamically: one worker, 2 threads, prefetch_count=4, a backend leasing 4 tasks and then failing to deserialize one — 4 tasks leased, 0 acknowledged, worker exited 0 before the loudness fix.
Options
- add a backend
release()/unacquire call that returns a leased task to READY (the inverse of the lease the acquire script takes), and call it for the buffered-and-unstarted remainder on a fetch failure - or lease only after the reply is fully deserialized (changes the atomicity story: the script must not move tasks to the running set until the client has them)
- or make the reaper hand back rather than fail on the first expiry, if a retry-once policy is acceptable
Related: the same "leased but never executed" window exists for a hard kill, which is accepted behaviour (the reaper covers it); this issue is about the error path where the run continues.
- Lenguaje dominante
- Python
- Estrellas
- 21
- Forks
- 2
- Merge medio
- 12 h 39 min
- PR fusionados (30 d)
- 27
Preparar el entorno
- Sin Dockerfile ni archivo de Docker Compose
- Sin plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Issues similares
-
Uninstall guide omits the CloudWatch log groups AWS CodeBuild creates for useCodeBuild pipelinesAbiertobug documentation
Dificultad 1/5 1-3 horas Aptitud para principiantes 88/100
awslabs/visual-asset-management-system#552 ·
Los mantenedores suelen responder en 1 día
-
netbox status: needs triage type: documentation
Dificultad 1/5 Menos de una hora Aptitud para principiantes 76/100
netbox-community/netbox#23460 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 75/100
awslabs/threat-composer#362 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
Los mantenedores suelen responder en 1 día
-
bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
ansys/pyansys-common-mcp#213 ·
Los mantenedores suelen responder en 3 días