feat(jobs): exports and seeding as queue jobs with a `jobs` table (no workflow engine)
維護者通常 1 天內回覆
還沒有人認領這個 Issue。
評估
- 難度
- 5/5
- 預估耗時
- 一週以上
- 新手友好度
- 20/100
- Issue 類型
- 功能
- 描述清晰度
- 描述清楚
- 活躍度
- 活躍
- 技術堆疊
- postgresql, rust
研究方向
Read ADR-0071, especially Option D and the round-29 note, alongside ADR-0046 and ADR-0073. Then inspect the migrations under crates/thrust-adapters-postgres/migrations/ and the worker contract test at crates/thrust-contract-tests/tests/worker_only.rs:203-226. Done means resumable chunked export and workspace seeding work on both worker-cf and native-pg, and the listed ADR, grep, and oracle checks pass.
由索引模型根據 Issue 內容生成。
描述
Context
Long jobs were planned on a WorkflowEngine port: Cloudflare Workflows through the JavaScript shim crates/thrust-worker/entry.mjs:1-26 (a Workflow class that calls back into Rust over the SELF service binding, :20-23; bindings at crates/thrust-worker/wrangler.toml:74-83), and a Postgres-backed step runner in the native binary (ADR-0071 Option B, lines 33, 60, 116). The port exists only in the plan; the code has a probe, worker_bindings_workflow_via_wasm_bindgen_and_js_shim (crates/thrust-contract-tests/tests/worker_only.rs:211-212). ADR-0073 maps the exports (issue_export_task, analytic_plot_export) to Workflows and workspace_seed to "W (or Q)" (lines 72-73, 82); exports are CSV or JSON with links valid 24 h and a size ceiling starting at 100 MB (line 122; Q-0081.2, Q-0081.3, Q-0073.3). #43 removes the shim, the Workflow binding and SELF.
Problem
- FR-05: the Workflows adapter is the only JavaScript in the Worker, and its exit plan waits on a workers-rs Workflow API with no date (ADR-0071 round 26, line 122). The self-hosted engine is net-new code where "we own correctness (timers, leader election, replay determinism)" (line 89).
- At this scale a queue plus a state table covers the job catalogue. Workers allow 15 minutes of wall time per queue-consumer or cron invocation; CPU per request defaults to 30 s and can be raised to 5 minutes with
limits.cpu_ms(workers/platform/limits). A cron that fires more often than hourly gets 30 s of CPU, so job steps belong in the consumer, not in the cron. - Seeding sits on the critical path earlier than M3:
POST /api/workspaces/dispatchesworkspace_seed, which creates a bot user and a sample project with members, states, labels and work items (research/0003lines 397-401), and that handler (S14) is in the 48-route slice (line 796).
Proposal
- A
jobstable incrates/thrust-adapters-postgres/migrations/:id,workspace_id,kind(export.work_items_csv,workspace.seed, ...),state(queued,running,succeeded,failed),cursor(resume point),attempts,result_key(aBlobStorekey),error,created_by, timestamps,expires_at. - Start: a use-case inserts the job row and a
job.requestedoutbox envelope in one transaction (ADR-0046), so the existing relay, queue and sweeper deliver it on both targets. - Step: the consumer claims the job, processes one bounded chunk, writes the part to
BlobStore, advancescursorandstatein one transaction, then enqueues the next step. Steps are idempotent on(job_id, cursor); the sweeper re-enqueuesrunningjobs whoseupdated_atis stale. - This is ADR-0071's Option D (queue-chained steps, state in the database), adopted in the ADR-0071 round-29 note; its main objection, matching Cloudflare Workflows semantics, goes away with the port. ADR-0073 rows move from W to Q.
- Seeding: a
workspace.seedjob reproduces Plane's seed. Until it lands, #44 records whether the slice seeds synchronously or deviates.
Acceptance criteria
- ADR-0071's reference workflow, restated as a job (export a 100k-work-item project to CSV in
BlobStore, Confirmation line 77), completes onworker-cfandnative-pg, with no step over the Worker's configured CPU limit on the real platform. - A contract test fails one export step with a transient error; the job resumes from its cursor, and the output equals that of an uninterrupted run (no missing or duplicated rows). Control: advancing the cursor outside the step's transaction makes it fail.
- After a workspace create, the seeded projects and members match the oracle's shapes on the routes in the acceptance list, or the deviation is recorded in
DEVIATIONS.md. -
grep -rnE "WorkflowEngine|WORKFLOW|ThrustWorkflow" cratesreturns nothing. - ADR-0073's rows for exports and seeding read Q; Q-0073.3 records the ceiling.
References
- Review: .agents/plans/contrail/reviews/2026-10-19-fable-architecture-review.md (FR-05)
crates/thrust-worker/entry.mjs:1-26;crates/thrust-worker/wrangler.toml:74-83;crates/thrust-contract-tests/tests/worker_only.rs:203-226- ADR-0071 (lines 33, 60, 77, 89, 96-99, 116, 122; round-29 note), ADR-0073 (lines 72-73, 82, 122), ADR-0046, ADR-0081
research/0003-compat-plane-inventory.mdlines 397-401, 796- Cloudflare docs: workers/platform/limits (wall time and CPU per invocation type)
- Q-0073.3, Q-0081.2, Q-0081.3, Q-0071.1
Blocked by: #43, #46
Finding: FR-05
- 主要語言
- Rust
- 星號
- 0
- 分支
- 0
- 平均合併
- 1 小時 35 分鐘
- 30 天內合併 PR
- 67
環境準備
- 沒有 Dockerfile 或 Docker Compose 檔案
- 沒有 Pull Request 範本
- 閱讀貢獻指南
從這裡開始
- 先讀完整個 Issue,再讀專案的貢獻指南。
- 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
- Fork 儲存庫,在一個分支上完成修改。
- 送出 Pull Request,並在描述裡引用這個 Issue 編號。
aRustyDev/thrust 的其他 Issue
-
area:process prio:p1 type:security
難度 4/5 3-5 天 新手友好度 48/100
維護者通常 1 天內回覆
-
area:process bug prio:p1
難度 4/5 3-5 天 新手友好度 38/100
維護者通常 1 天內回覆
-
area:process prio:p1
難度 4/5 3-5 天 新手友好度 55/100
維護者通常 1 天內回覆
-
area:process prio:p1
難度 4/5 1-2 天 新手友好度 55/100
維護者通常 1 天內回覆
-
area:process prio:p1
難度 4/5 3-5 天 新手友好度 56/100
維護者通常 1 天內回覆
相似的 Issue
-
Panic on live viewport resize when unstyled elements exist in display: none subtree可能已有人在做 @GoCoder7 今天認領。 未關閉
難度 2/5 1-3 小時 新手友好度 78/100
DioxusLabs/blitz#1488 ·
維護者通常 1 天內回覆
-
🏷️ good first issue enhancement
難度 2/5 1-3 小時 新手友好度 74/100
維護者通常 1 天內回覆
-
S-Needs triage
難度 2/5 1-3 小時 新手友好度 68/100
維護者通常 2 天內回覆
-
難度 2/5 1-3 小時 新手友好度 77/100
vectordotdev/vrl#2042 ·
維護者通常 2 天內回覆
-
難度 2/5 1-3 小時 新手友好度 62/100
維護者通常 1 天內回覆