Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

A failed fetch leaves the acquired batch to the reaper

Chiusa
  • Qualcosa non va?
#63 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
35/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
lua, python, redis
Ambito
backend, databases, distributed-systems

Direzione di ricerca

Start with acquire.lua and trace how leased tasks are handled when TaskResult deserialization fails. Then inspect maintain_worker_pool and the reaper's lease-expiry behavior. Done means fetch failures no longer silently lose leased tasks and the intended worker-drain behavior remains intact.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Deferred from the task-prefetching review (crew run): the error path is now loud and the worker exits non-zero, but returning the tasks is the larger design piece.

What

acquire.lua leases every task of a batch (ZADD into the running set with the lease deadline, task data rewritten RUNNING) before the reply is deserialized. If the fetcher then raises — a poison payload that fails TaskResult reconstruction, a renamed or removed task function, a broker error mid-reply — the whole leased batch is discarded:

  • the batch is never buffered, never executed, and never acknowledged
  • the reaper moves it to the failed history after lease_ttl (default 1 hour) and never back to READY, so the tasks are lost unless an operator requeues them
  • with --exit-empty, dead workers are continued by maintain_worker_pool, so a broker outage across the fetchers can end the whole drain run as if the queue were empty

Reproduced dynamically: one worker, 2 threads, prefetch_count=4, a backend leasing 4 tasks and then failing to deserialize one — 4 tasks leased, 0 acknowledged, worker exited 0 before the loudness fix.

Options

  • add a backend release()/unacquire call that returns a leased task to READY (the inverse of the lease the acquire script takes), and call it for the buffered-and-unstarted remainder on a fetch failure
  • or lease only after the reply is fully deserialized (changes the atomicity story: the script must not move tasks to the running set until the client has them)
  • or make the reaper hand back rather than fail on the first expiry, if a retry-once policy is acceptable

Related: the same "leased but never executed" window exists for a hard kill, which is accepted behaviour (the reaper covers it); this issue is about the error path where the run continues.

Lingua principale
Python
Stelle
21
Fork
2
Merge medio
12h 39m
PR unite (30g)
27

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.