feat(pipeline): 派发决策上移 plaita console——keeper 退化为探测器+收尸者,本地/远端多 worker 统一领单
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 20/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- python, redis
Research direction
Start by reading flows/pipeline_bridge.py, plaita/server/task_queue.py, plaita/server/execution_lease.py, and plaita/server/registry.py, then trace the existing pipeline dispatch path and the console _enqueue() entry point in plaita-console/backend/api/executions.py. The issue defines four phases and acceptance checks for local-mode compatibility, claim-mode delegation, duplicate prevention, and rollback; this is broad cross-system design work rather than a self-contained first change.
Written by the indexing model from the issue text.
Description
优先级 P1 · 依赖:无
基线 HEAD = 16fa769(git -C issue-keeper rev-parse --short origin/main)
Summary
把「派发决策」从 keeper 上移到 plaita console,keeper 退化为「探测器 + 收尸者」,本地/远端各起多个 worker 向 console 领单。当前结构性缺口(亲读证据):
- 两个 keeper 各自扫自己那份绑定表,互不知情:本机
~/.issue-keeper/state.json与远端 tcloud_gz 的 state 完全独立;本机config.yaml的pipeline_max_in_flight/pipeline_priority_repos/author_daily_limit均为 per-keeper 本地配置。实证:本机排队 62 条(recursive 待派 27 / plaita 待派 18)同时远端九仓仅剩 1 条待派——一边积压一边空转。 - console 只存定义、不参与派发:
flows/pipeline_bridge.py:7注释原文「混合形态:定义与观测归 plaita-console,执行留本仓」;pipeline_bridge.py:11-15的降级链只解析 flow 定义,不含任何派发逻辑。 - plaita 侧分布式能力已齐备但未被派发层使用:
plaita/server/task_queue.py(Redis Stream 消费组抢单、DEFAULT_CLAIM_MIN_IDLE_MS=60_000、DEFAULT_MAX_DELIVERIES=5、DLQ)、plaita/server/execution_lease.py(DEFAULT_LEASE_TTL_SECONDS=120+ fencing token,跨机防双跑)、plaita/server/registry.py(worker 注册/30s 心跳)。console 已有发单入口plaita-console/backend/api/executions.py:148 _enqueue()→plaita/server/task_queue.enqueue_task()。
影响
- 产能浪费:本机 3 并发跑满时 62 条排队,远端 2 并发闲置——全仓吞吐被单机封顶。
- 运维耦合:改并发/优先级必须改每个 keeper 的本地 config 并重启 daemon(launchd
kickstart)——无法全局调节。 - 无法按容量避险:本机 2026-10-06 10:10 磁盘触底 0 字节(shell 写失败实证,见
~/.issue-keeper/pipeline/controller/rounds.log当日行)时,派发侧无从知晓并继续往本机压单。 - 跨机同仓双跑风险靠"绑定表切分"人工维持:
keeper-remote-worker-tcloud-gz记录「同一仓只归一侧」,一旦两边绑定表漂移即双跑(#136 家族同工作区互杀)。
建议
分四阶段(每阶段可停可回退):
- P0 迁移:plaita console 从本机迁至 tcloud_gz(当前 console 实测在本机 pid 94089 监听 127.0.0.1:8123;远端 8123 为 docker-proxy 非 console,远端经 ssh -R 隧道以
PLAITA_CONSOLE_URL=http://127.0.0.1:18123反连本机)。 - P1 全局可见(只读):console 汇总两台 state → 全局队列/worker 面板,不改编排。
- P2 队列化派发:新建
issue-intakeflow 承担派单;keeper 改为「扫 issue → 入队」与「run 终态 → 回评/关闭」;worker 改为「从 Stream 领单」。必须:新增SCHEDULER_MODE=local|claim开关与delegated_repos(让 keeper 对已委托给新系统的仓只扫不派),否则新旧两套会争抢同一 issue。 - P3 容量感知 + 收尸 flow 化:worker 上报负荷(磁盘/内存/在跑数),不足者自动摘除;
issue-reaperflow 接管终态处理。
验收条件(逐条可断言):
SCHEDULER_MODE=local时行为与当前完全一致(回归:既有 pipeline 全链跑通、台账格式不变);SCHEDULER_MODE=claim且delegated_repos=[X]时,keeper 对 X 仓不派发(日志断言零派发)、X 仓的单由 worker 领单执行,且同一 issue 不出现两次执行(execution_lease 持锁断言);- 并行验证期(金丝雀):新系统只接指定仓,旧流程跑其余仓,两套台账可对照且互不污染(队列名隔离断言);
- 切换仅需改配置(不改代码、不重启 worker 之外无其它动作);回退=改回配置。
边界
- 不做:不改
issue-pipelineflow 主体(60 节点复用,仅入口改为领单);不做多租户收窄(README 已标该边界)。 - 并行验证(jeffkit 2026-10-06 要求):新系统跑通前不切流;验证期采用「影子模式(只算不发)→ 金丝雀(按仓)→ 全量」三步,禁止全量双跑同一 issue。
- 与 plaita 配套单(console 侧 intake/heartbeat/reaper flow 与容量上报)互相引用。
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from jeffkit/issue-keeper
-
Difficulty 4/5 3-5 days Newbie friendliness 15/100
jeffkit/issue-keeper#22 · 3 comments ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
jeffkit/issue-keeper#21 · 3 comments ·
-
Difficulty 4/5 3-5 days Newbie friendliness 58/100
jeffkit/issue-keeper#19 · 1 comment ·
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
jeffkit/issue-keeper#17 · 6 comments ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
jeffkit/issue-keeper#11 · 5 comments ·
All issues in jeffkit/issue-keeper
Similar issues
-
documentation pkg: sphinx-needs
Difficulty 1/5 Under an hour Newbie friendliness 82/100
useblocks/sphinx-needs#2255 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 60/100
home-assistant/core#186538 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
btclib-org/btclib-secp256k1#1181 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
pyrite-wiki/pyrite#917 ·
Maintainers usually reply within 1 day