A stored payload that does not deserialize keeps the reaper in a loop
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 55/100
- Issue 类型
- 缺陷
- 描述清晰度
- 描述清楚
- 活跃度
- 活跃
调研方向
Start in threadmill/backends/redis.py at RedisBroker._reap_task and compare its deserialization failure path with the replaced Lua reaper's finalize step. Trace the raw JSON fields and verify that an undeserializable task is finalized as FAILED, publishes a result, and is not re-claimed indefinitely.
由索引模型根据 Issue 内容生成。
描述
Found by the SuperJoe review of the lease-expiry retry patch (4ec9068..d5a1fdb).
RedisBroker._reap_task deserializes the stored task before it decides the fate of the task:
# threadmill/backends/redis.py
data = self.backend.client.hget(task_key, "data")
if data is None:
logger.warning("Claimed task %r has no task data; skipping", task_id)
return
task_result = self.backend.deserialize_task_result(data)
deserialize_task_result imports the task and its retry callback from the serialized dotted paths. A deployment can remove or rename the callback while the task waits in the queue. Then the import fails and the call raises on every pass. The surrounding logger.exception only writes a log record. The claim stays:
- The reaper claims the task again every
CLAIM_TTLand the same error occurs. - The task never reaches
FAILEDand never publishes a result. - Only expiry of the task hash or a late acknowledgement removes the running entry.
The replaced Lua reaper decoded the same payload with cjson and finalized it as FAILED without any import. This error is a regression from that rewrite.
Suggested correction: add a salvage branch to the reap path. The branch finalizes a payload that the code cannot deserialize. The raw JSON already carries status, errors, and the task fields. Patch it and publish it as FAILED. A small Lua script that copies the finalize step of the old reaper can do this. Then the behavior matches the pre-rewrite state and the retry decision does not return to Lua.
- 主要语言
- Python
- 星标
- 19
- 派生
- 1
- 平均合并
- 9 小时 46 分钟
- 30 天内合并 PR
- 17
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
codingjoe/threadmill 的其他 Issue
-
bug real side quest
难度 5/5 一周以上 新手友好度 35/100
codingjoe/threadmill#78 · 1 条评论 ·
维护者通常 1 天内回复
-
perf real
难度 4/5 3-5 天 新手友好度 55/100
codingjoe/threadmill#74 ·
维护者通常 1 天内回复
-
bug real
难度 4/5 3-5 天 新手友好度 45/100
codingjoe/threadmill#75 · 1 条评论 ·
维护者通常 1 天内回复
-
bug real
难度 4/5 3-5 天 新手友好度 64/100
codingjoe/threadmill#71 ·
维护者通常 1 天内回复
-
perf real
难度 5/5 一周以上 新手友好度 35/100
codingjoe/threadmill#67 ·
维护者通常 1 天内回复
查看 codingjoe/threadmill 的全部 Issue
相似的 Issue
-
难度 1/5 1 小时以内 新手友好度 85/100
Vector35/community-plugins#376 ·
-
难度 2/5 1-3 小时 新手友好度 68/100
py-econometrics/pyfixest#1883 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 65/100
ietf-tools/rfc2html#81 ·
-
难度 1/5 1 小时以内 新手友好度 88/100
mysql/mysql-operator#60 ·
-
Python: Bug: split_plaintext_paragraph / split_markdown_paragraph can return a chunk larger than max_tokens可能已有人在做 @xThreeh 今天认领。 未关闭python triage
难度 2/5 1-3 小时 新手友好度 75/100
microsoft/semantic-kernel#14566 ·
维护者通常 4 天内回复