A failed fetch leaves the acquired batch to the reaper
維護者通常 1 天內回覆
還沒有人認領這個 Issue。
評估
- 難度
- 5/5
- 預估耗時
- 一週以上
- 新手友好度
- 35/100
- Issue 類型
- 缺陷
- 描述清晰度
- 基本清楚
- 活躍度
- 活躍
研究方向
Start with acquire.lua and trace how leased tasks are handled when TaskResult deserialization fails. Then inspect maintain_worker_pool and the reaper's lease-expiry behavior. Done means fetch failures no longer silently lose leased tasks and the intended worker-drain behavior remains intact.
由索引模型根據 Issue 內容生成。
描述
Deferred from the task-prefetching review (crew run): the error path is now loud and the worker exits non-zero, but returning the tasks is the larger design piece.
What
acquire.lua leases every task of a batch (ZADD into the running set with the lease deadline, task data rewritten RUNNING) before the reply is deserialized. If the fetcher then raises — a poison payload that fails TaskResult reconstruction, a renamed or removed task function, a broker error mid-reply — the whole leased batch is discarded:
- the batch is never buffered, never executed, and never acknowledged
- the reaper moves it to the failed history after
lease_ttl(default 1 hour) and never back to READY, so the tasks are lost unless an operator requeues them - with
--exit-empty, dead workers arecontinued bymaintain_worker_pool, so a broker outage across the fetchers can end the whole drain run as if the queue were empty
Reproduced dynamically: one worker, 2 threads, prefetch_count=4, a backend leasing 4 tasks and then failing to deserialize one — 4 tasks leased, 0 acknowledged, worker exited 0 before the loudness fix.
Options
- add a backend
release()/unacquire call that returns a leased task to READY (the inverse of the lease the acquire script takes), and call it for the buffered-and-unstarted remainder on a fetch failure - or lease only after the reply is fully deserialized (changes the atomicity story: the script must not move tasks to the running set until the client has them)
- or make the reaper hand back rather than fail on the first expiry, if a retry-once policy is acceptable
Related: the same "leased but never executed" window exists for a hard kill, which is accepted behaviour (the reaper covers it); this issue is about the error path where the run continues.
- 主要語言
- Python
- 星號
- 21
- 分支
- 2
- 平均合併
- 12 小時 39 分鐘
- 30 天內合併 PR
- 27
環境準備
- 沒有 Dockerfile 或 Docker Compose 檔案
- 沒有 Pull Request 範本
- 閱讀貢獻指南
從這裡開始
- 先讀完整個 Issue，再讀專案的貢獻指南。
- 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
- Fork 儲存庫，在一個分支上完成修改。
- 送出 Pull Request，並在描述裡引用這個 Issue 編號。
相似的 Issue
-
難度 2/5 1-3 小時 新手友好度 90/100
SublimeText/AdvancedNewFile#211 ·
-
難度 2/5 1-3 小時 新手友好度 73/100
-
難度 2/5 1-3 小時 新手友好度 83/100
Imbad0202/academic-research-skills#987 ·
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 62/100
-
Pull-through from gcr.io returns 500 when pulling using podman/buildah可能已有人在做 關聯的 PR 仍在進行中或已合併。 未關閉Issue Triage-Needed
難度 2/5 1-3 小時 新手友好度 68/100
pulp/pulp_container#2581 ·
維護者通常 1 天內回覆