feat(jobs): exports and seeding as queue jobs with a `jobs` table (no workflow engine)
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
- 难度
- 5/5
- 预计耗时
- 一周以上
- 新手友好度
- 20/100
- Issue 类型
- 功能
- 描述清晰度
- 描述清楚
- 活跃度
- 活跃
- 技术栈
- postgresql, rust
调研方向
Read ADR-0071, especially Option D and the round-29 note, alongside ADR-0046 and ADR-0073. Then inspect the migrations under crates/thrust-adapters-postgres/migrations/ and the worker contract test at crates/thrust-contract-tests/tests/worker_only.rs:203-226. Done means resumable chunked export and workspace seeding work on both worker-cf and native-pg, and the listed ADR, grep, and oracle checks pass.
由索引模型根据 Issue 内容生成。
描述
Context
Long jobs were planned on a WorkflowEngine port: Cloudflare Workflows through the JavaScript shim crates/thrust-worker/entry.mjs:1-26 (a Workflow class that calls back into Rust over the SELF service binding, :20-23; bindings at crates/thrust-worker/wrangler.toml:74-83), and a Postgres-backed step runner in the native binary (ADR-0071 Option B, lines 33, 60, 116). The port exists only in the plan; the code has a probe, worker_bindings_workflow_via_wasm_bindgen_and_js_shim (crates/thrust-contract-tests/tests/worker_only.rs:211-212). ADR-0073 maps the exports (issue_export_task, analytic_plot_export) to Workflows and workspace_seed to "W (or Q)" (lines 72-73, 82); exports are CSV or JSON with links valid 24 h and a size ceiling starting at 100 MB (line 122; Q-0081.2, Q-0081.3, Q-0073.3). #43 removes the shim, the Workflow binding and SELF.
Problem
- FR-05: the Workflows adapter is the only JavaScript in the Worker, and its exit plan waits on a workers-rs Workflow API with no date (ADR-0071 round 26, line 122). The self-hosted engine is net-new code where "we own correctness (timers, leader election, replay determinism)" (line 89).
- At this scale a queue plus a state table covers the job catalogue. Workers allow 15 minutes of wall time per queue-consumer or cron invocation; CPU per request defaults to 30 s and can be raised to 5 minutes with
limits.cpu_ms(workers/platform/limits). A cron that fires more often than hourly gets 30 s of CPU, so job steps belong in the consumer, not in the cron. - Seeding sits on the critical path earlier than M3:
POST /api/workspaces/dispatchesworkspace_seed, which creates a bot user and a sample project with members, states, labels and work items (research/0003lines 397-401), and that handler (S14) is in the 48-route slice (line 796).
Proposal
- A
jobstable incrates/thrust-adapters-postgres/migrations/:id,workspace_id,kind(export.work_items_csv,workspace.seed, ...),state(queued,running,succeeded,failed),cursor(resume point),attempts,result_key(aBlobStorekey),error,created_by, timestamps,expires_at. - Start: a use-case inserts the job row and a
job.requestedoutbox envelope in one transaction (ADR-0046), so the existing relay, queue and sweeper deliver it on both targets. - Step: the consumer claims the job, processes one bounded chunk, writes the part to
BlobStore, advancescursorandstatein one transaction, then enqueues the next step. Steps are idempotent on(job_id, cursor); the sweeper re-enqueuesrunningjobs whoseupdated_atis stale. - This is ADR-0071's Option D (queue-chained steps, state in the database), adopted in the ADR-0071 round-29 note; its main objection, matching Cloudflare Workflows semantics, goes away with the port. ADR-0073 rows move from W to Q.
- Seeding: a
workspace.seedjob reproduces Plane's seed. Until it lands, #44 records whether the slice seeds synchronously or deviates.
Acceptance criteria
- ADR-0071's reference workflow, restated as a job (export a 100k-work-item project to CSV in
BlobStore, Confirmation line 77), completes onworker-cfandnative-pg, with no step over the Worker's configured CPU limit on the real platform. - A contract test fails one export step with a transient error; the job resumes from its cursor, and the output equals that of an uninterrupted run (no missing or duplicated rows). Control: advancing the cursor outside the step's transaction makes it fail.
- After a workspace create, the seeded projects and members match the oracle's shapes on the routes in the acceptance list, or the deviation is recorded in
DEVIATIONS.md. -
grep -rnE "WorkflowEngine|WORKFLOW|ThrustWorkflow" cratesreturns nothing. - ADR-0073's rows for exports and seeding read Q; Q-0073.3 records the ceiling.
References
- Review: .agents/plans/contrail/reviews/2026-10-19-fable-architecture-review.md (FR-05)
crates/thrust-worker/entry.mjs:1-26;crates/thrust-worker/wrangler.toml:74-83;crates/thrust-contract-tests/tests/worker_only.rs:203-226- ADR-0071 (lines 33, 60, 77, 89, 96-99, 116, 122; round-29 note), ADR-0073 (lines 72-73, 82, 122), ADR-0046, ADR-0081
research/0003-compat-plane-inventory.mdlines 397-401, 796- Cloudflare docs: workers/platform/limits (wall time and CPU per invocation type)
- Q-0073.3, Q-0081.2, Q-0081.3, Q-0071.1
Blocked by: #43, #46
Finding: FR-05
- 主要语言
- Rust
- 星标
- 0
- 派生
- 0
- 平均合并
- 1 小时 35 分钟
- 30 天内合并 PR
- 67
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
aRustyDev/thrust 的其他 Issue
-
area:process prio:p1 type:security
难度 4/5 3-5 天 新手友好度 48/100
维护者通常 1 天内回复
-
area:process bug prio:p1
难度 4/5 3-5 天 新手友好度 38/100
维护者通常 1 天内回复
-
area:process prio:p1
难度 4/5 3-5 天 新手友好度 55/100
维护者通常 1 天内回复
-
area:process prio:p1
难度 4/5 1-2 天 新手友好度 55/100
维护者通常 1 天内回复
-
area:process prio:p1
难度 4/5 3-5 天 新手友好度 56/100
维护者通常 1 天内回复
相似的 Issue
-
yara-x fails to build with rsa 0.10.0: unresolved import rsa::Pkcs1v15Sign可能已有人在做 @CfirTsabari 今天认领。 未关闭
难度 1/5 1 小时以内 新手友好度 87/100
VirusTotal/yara-x#865 ·
维护者通常 1 天内回复
-
good first issue help wanted program
难度 2/5 1-3 小时 新手友好度 83/100
hffgzktbcc-blip/ounce-core#297 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 72/100
nautechsystems/nautilus_trader#5631 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 69/100
bytecodealliance/rustix#1736 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 78/100
longbridge/gpui-kit#3871 ·
维护者通常 1 天内回复