Hacktoberfest 2026: as issues que os mantenedores marcaram para outubro, abertas e boas para iniciantes. Ver issues do Hacktoberfest

A failed fetch leaves the acquired batch to the reaper

Fechada
  • Algo errado?
#63 0 comentários 0 reações 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 1 dia

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
5/5
Tempo estimado
Mais de uma semana
Facilidade para iniciantes
35/100
Tipo de issue
Bug
Clareza
Razoavelmente clara
Status de atividade
Ativa
Stack de tecnologia
lua, python, redis
Domínio
backend, databases, distributed-systems

Direção de pesquisa

Start with acquire.lua and trace how leased tasks are handled when TaskResult deserialization fails. Then inspect maintain_worker_pool and the reaper's lease-expiry behavior. Done means fetch failures no longer silently lose leased tasks and the intended worker-drain behavior remains intact.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

Deferred from the task-prefetching review (crew run): the error path is now loud and the worker exits non-zero, but returning the tasks is the larger design piece.

What

acquire.lua leases every task of a batch (ZADD into the running set with the lease deadline, task data rewritten RUNNING) before the reply is deserialized. If the fetcher then raises — a poison payload that fails TaskResult reconstruction, a renamed or removed task function, a broker error mid-reply — the whole leased batch is discarded:

  • the batch is never buffered, never executed, and never acknowledged
  • the reaper moves it to the failed history after lease_ttl (default 1 hour) and never back to READY, so the tasks are lost unless an operator requeues them
  • with --exit-empty, dead workers are continued by maintain_worker_pool, so a broker outage across the fetchers can end the whole drain run as if the queue were empty

Reproduced dynamically: one worker, 2 threads, prefetch_count=4, a backend leasing 4 tasks and then failing to deserialize one — 4 tasks leased, 0 acknowledged, worker exited 0 before the loudness fix.

Options

  • add a backend release()/unacquire call that returns a leased task to READY (the inverse of the lease the acquire script takes), and call it for the buffered-and-unstarted remainder on a fetch failure
  • or lease only after the reply is fully deserialized (changes the atomicity story: the script must not move tasks to the running set until the client has them)
  • or make the reaper hand back rather than fail on the first expiry, if a retry-once policy is acceptable

Related: the same "leased but never executed" window exists for a hard kill, which is accepted behaviour (the reaper covers it); this issue is about the error path where the run continues.

Linguagem predominante
Python
Estrelas
21
Forks
2
Merge médio
12h 39min
PRs com merge (30d)
27

Preparar o ambiente

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Issues semelhantes

Mais issues de Python

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.