Silently dropped task ids when the task hash is missing in acquire.lua and mover.lua
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Active
- Tech stack
- lua, redis
- Domain
- backend, distributed-systems
Research direction
Start by reading threadmill/backends/lua/acquire.lua and threadmill/backends/lua/mover.lua, then reproduce the QED case by deleting a task hash while its id remains queued. Review how result rows are serialized and consumed by peek() and the inspector. The issue is done only after the failure policy is decided, task ids are no longer silently lost, and regression coverage verifies the chosen behavior.
Written by the indexing model from the issue text.
Description
Split out of #48 during the fair multi-queue scheduling change (codingjoe-fair-multi-queue-scheduling).
defer: Silent task loss when a ready/deferred set member's task hash is gone (acquire.lua drops the id, mover.lua drops a due id whose score is gone) with no result row written. Separate durability bug; the sets have no TTL. [threadmill/backends/lua/acquire.lua, threadmill/backends/lua/mover.lua]
Problem
ZPOPMIN removes the member before the data hash is read, so when HGET returns nothing (hash gone, or cjson.decode fails) the loop moves on and the task id is left nowhere: not in the ready set, not in the running set, and no result row is written.
mover.lua has the same shape: it drops a due id from the deferred set when HGET score returns nothing, again without a result row.
The ready and deferred sets have no TTL, while the task hash expires after lease_ttl * 3 + result_ttl (27 h with the defaults), so a backlog or a worker outage longer than that silently loses tasks instead of failing them. That conflicts with the project's Durability and Consistency design principles.
Open questions
- Should the expired hash be reported as a FAILED result (with what payload, given the data is gone), or should the ready/deferred sets be bounded or refreshed instead?
- Is a synthetic result row even deserializable by
peek()and the inspector, which expect a serializedTaskResult?
QED
- Enqueue a task.
- Delete its hash key (
{prefix}:task:{id}) while it stays in the ready set. acquire()— the id vanishes with no trace.
- Dominant language
- Python
- Stars
- 12
- Forks
- 1
- Avg merge
- 19h 11m
- Merged PRs (30d)
- 12
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from codingjoe/threadmill
-
Difficulty 2/5 Half a day Newbie friendliness 84/100
codingjoe/threadmill#52 ·
-
WorkerProcess passes None poll intervals to the backend, breaking acquire in the consumer thread Open
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
codingjoe/threadmill#53 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
codingjoe/threadmill#51 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
codingjoe/threadmill#47 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 68/100
codingjoe/threadmill#55 · 1 comment ·
All issues in codingjoe/threadmill
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
anthropics/skills#1811 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
speaches-ai/speaches#678 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
datalayer/mcp-compose#42 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
conda-forge/spacy-feedstock#177 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
UKGovernmentBEIS/inspect_evals#2523 ·