Workflow dep resolution leaves `scheduled_at` stale, breaking queue delay monitoring
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 52/100
- Issue 类型
- 缺陷
- 描述清晰度
- 基本清楚
- 活跃度
- 冷清
- 技术栈
- go, postgresql
调研方向
Locate the WorkflowStageJobs and WorkflowStageJobsByIDMany queries and inspect the jobs_to_make_available CTE plus its UPDATE of river_job. Confirm the behavior for jobs becoming available versus remaining scheduled, then choose and implement the agreed timestamp approach; done means dependency-resolved available jobs report an accurate queue-delay timestamp without breaking scheduled jobs.
由索引模型根据 Issue 内容生成。
描述
Description
When WorkflowStageJobs / WorkflowStageJobsByIDMany resolve dependencies and transition a job from pending to available, the scheduled_at column is not updated. It retains its original value from insertion time, which can be hours or months old for long-running workflows.
The UPDATE in both queries only sets state and metadata.workflow_staged_at:
UPDATE river_job
SET
state = jobs_to_make_available.new_state,
metadata = jsonb_set(metadata, '{workflow_staged_at}'::text[], $1::jsonb, true)
FROM jobs_to_make_available
WHERE river_job.id = jobs_to_make_available.id
The jobs_to_make_available CTE already reads scheduled_at to decide the target state (available if scheduled_at <= now() + 5s, otherwise scheduled), so by the time the UPDATE executes, the original scheduled_at value has served its purpose.
Impact
Any monitoring that uses NOW() - scheduled_at on available jobs to measure queue delay will report wildly inflated values for dependency-resolved workflow jobs. For workflows where deps take hours or months to resolve, this produces false alarms on queue health metrics.
Current workaround
We discovered that workflow_staged_at is already stamped in metadata during dep resolution, so we use it as a fallback in our metrics query:
MAX(
CASE
WHEN metadata ? 'workflow_staged_at'
THEN NOW() - (metadata->>'workflow_staged_at')::timestamptz
ELSE NOW() - scheduled_at
END
) as oldest_delay
This works but requires casting a JSONB string to timestamptz in an aggregate query, which is less ergonomic than using the native scheduled_at column directly.
Proposed solutions
Either of these would address the problem:
-
Update
scheduled_at = now()inWorkflowStageJobs/WorkflowStageJobsByIDManywhen transitioning jobs toavailable. This makesscheduled_ataccurately reflect when the job became eligible for pickup, consistent with how non-workflow jobs behave. For jobs transitioning toscheduled(because theirscheduled_atis still in the future), no change is needed —scheduled_atis already correct. -
Add a first-class
available_atcolumn toriver_jobthat records when a job entered theavailablestate, regardless of how it got there (direct insert, scheduled time reached, or workflow dep resolution). This would give monitoring queries a reliable, indexed timestamp without relying onscheduled_atsemantics or JSONB metadata. It would also benefit non-workflow use cases like jobs inserted withPending: truethat are later moved toavailableby application code.
Environment
- River Pro v0.22.0
- PostgreSQL
- 主要语言
- Go
- 星标
- 5.7k
- 派生
- 179
- 平均合并
- 2 天 19 小时
- 30 天内合并 PR
- 12
环境准备
我们还没有检查这个项目的环境配置文件。先看它的 README,通用步骤见我们的新手贡献指南。
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
riverqueue/river 的其他 Issue
-
难度 5/5 一周以上 新手友好度 45/100
riverqueue/river#1358 · 1 条评论 ·
维护者通常 1 天内回复
-
难度 4/5 3-5 天 新手友好度 35/100
riverqueue/river#1258 · 7 条评论 ·
维护者通常 1 天内回复
-
难度 4/5 3-5 天 新手友好度 45/100
riverqueue/river#1225 · 14 条评论 ·
维护者通常 1 天内回复
-
难度 4/5 3-5 天 新手友好度 52/100
riverqueue/river#1183 · 2 个 reaction ·
维护者通常 1 天内回复
-
难度 5/5 一周以上 新手友好度 30/100
riverqueue/river#1167 · 2 条评论 ·
维护者通常 1 天内回复
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 78/100
-
难度 1/5 1 小时以内 新手友好度 65/100
521xueweihan/HelloGitHub#3789 ·
-
难度 2/5 1-3 小时 新手友好度 88/100
维护者通常 12 天内回复
-
stage-fail
难度 2/5 1-3 小时 新手友好度 78/100
siyuan-note/bazaar#2282 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 78/100
openshift/kube-compare#307 ·
维护者通常 1 天内回复