delay_service 已消费的延迟任务只在内存线程 sleep——进程重启即丢唤醒,执行永久 suspended
Nobody has claimed this yet.
- #17 by @jeffkit — closed without merging
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 48/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- python
- Domain
- backend, distributed-systems
Research direction
Read base_service.py and delay_service.py to trace how delay tasks move from the Redis queue into the in-memory trigger loop, then compare the recovery approach in schedule_service. Implement persistent pending-task tracking and startup recovery for due and future tasks, and verify that tasks survive a delay_service restart without remaining suspended.
Written by the indexing model from the issue text.
Description
现象
delay 节点的执行链路存在一个可靠性缺口:worker 挂起时把任务 RPUSH 到 plaita:delay:queue,delay_service BLPOP 消费后在内存线程里 sleep 到触发时刻(base_service.py ThreadPoolExecutor + delay_service.py 的 trigger 循环)。如果 delay_service 在「已消费、未触发」窗口内重启/被杀,任务丢失且无任何补偿——对应执行永久停留在 suspended,只能人工 resume。
证据
- 任务里已经带了
planned_trigger_timestamp(delay_service 构造 service_config 时写入),但没有任何启动扫描/补偿逻辑读它 - 对比:schedule_service 有 next_run_at 落 Redis + 每秒扫描的自愈设计;delay 队列的任务一旦出队就只活在内存里
- E2E 混沌演练中确认:ext-services 容器在 delay 执行中途被杀 → 该执行无法恢复(这是设计缺口,不是回归,故未做成断言)
建议
延迟任务出队后把 {execution_id, planned_trigger_timestamp, event_type, task_config} 落 Redis(如 plaita:delay:pending hash),触发后删除;delay_service 启动时扫描该结构,对已到期的任务直接补触发、未到期的重新调度。这样重启窗口内的延迟任务可恢复。
影响
多容器部署(docker-compose / k8s)下 ext-services 的例行重启/滚动更新都会触发此缺口。当前 5 容器 E2E 拓扑里它被 chaos 脚本刻意绕开(不动 ext-services),但生产上绕不开。
- Dominant language
- Python
- Stars
- 0
- Forks
- 1
- Avg merge
- 3h 53m
- Merged PRs (30d)
- 3
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from jeffkit/plaita
-
Difficulty 1/5 Under an hour Newbie friendliness 92/100
-
Difficulty 4/5 3-5 days Newbie friendliness 49/100
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
Similar issues
-
HTML: <template> content is extracted as document textPossibly taken @ryanmeowy claimed this today. Openbug html
Difficulty 1/5 Under an hour Newbie friendliness 82/100
docling-project/docling#4714 · 2 comments ·
Maintainers usually reply within 1 day
-
[BUG] Qdrant RAG client applies score_threshold to raw cosine similarity, not the 0-1 score it returnsPossibly taken @roydonsequeira claimed this today. Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
ashhart/TensorFold#536 ·
Maintainers usually reply within 1 day
-
area/install-update comp/cli duplicate P2 python:uv sweeper:risk-compatibility type/bug
Difficulty 1/5 Under an hour Newbie friendliness 62/100
NousResearch/hermes-agent#135440 · 1 comment ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Deepak3699/Ai_Mentor#244 ·
Maintainers usually reply within 1 day