Scheduled-job skip blames an application operation when another scheduled job holds the lock
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
调研方向
从 internal/engine/schedule.go 中第343行附近的 skip() 以及第356行和第363-364行的锁检查开始;将 application lock 的处理与 schedule.lock 的失败路径进行比较,并查看 internal/engine/lock.go:96-97 以了解另一个锁持有者。跟踪所选 reason 如何到达 state file、notifier 和 ob status。完成的标准是:job 与 job 之间的冲突不会被报告为 application operation,并且两个可区分的情况都有覆盖。
由索引模型根据 Issue 内容生成。
描述
What happens
When a scheduled job stands aside because it could not take schedule.lock, it records:
an application operation is taking its lock
That names an application operation — a deploy, a job run, an exec — as the holder. In the common case it is another scheduled job, and no application operation is involved at all.
schedule.lock is taken by both:
- Application operations, but only around the atomic creation of the application lock (
internal/engine/lock.go:96-97), so for milliseconds. - Every scheduled job run, for as long as it needs (
internal/engine/schedule.go:362) — an exclusive job holds it for its whole run, a pinned one until its release lease exists.
So the message describes the rarer holder and omits the frequent one. An operator reading it is pointed at their own deploys and exec sessions, when nothing they do to those will change the outcome.
The runner already distinguishes the two cases it can name accurately:
stand_aside 'another run of this job is still in progress'— the per-job lock (schedule.go:356), accurate.skip 'an application operation holds the deploy lock'— the application lock file, tested directly (schedule.go:364), accurate.
It is only the flock on schedule.lock (schedule.go:363) that guesses, and guesses wrong.
skip() writes the reason it is handed and nothing else (schedule.go:343), so the misleading string is what lands in the state file, the notifier and ob status — there is no better-quality record behind it to fall back on.
Observed
A production application with sixteen scheduled jobs, several sharing cron boundaries (* * * * *, */2 * * * *, two firing together on :00):
schedule sync-sec-submissions skipped 5 firings in a row: an application operation is taking its lock ⚠
schedule sync-bls skipped 3 firings in a row: an application operation is taking its lock ⚠
schedule sync-cftc-cot skipped 3 firings in a row: an application operation is taking its lock ⚠
Ten of the sixteen showed that reason as their last outcome. This sent me looking at the operator's ob exec sessions, which had run in the same window and were the obvious suspect given the wording. They were not the cause: an exec holds schedule.lock only during application-lock creation, and had it been the application lock the reason would have been the other message. The real contention was job against job.
The diagnosis cost is the point — the message is confident and specific, and it is confidently pointing at the wrong thing.
Expected
The reason names what actually held the lock. The runner cannot see the holder directly, but it does not have to guess either: flock failing here means something holds it, and the two candidates are distinguishable at the point of failure — the application lock file's presence and age is already tested a line later.
Roughly: if the application lock exists and is within its TTL, say so; otherwise say another scheduled job holds it. Failing that, a reason that does not assert a cause it has not established would still be an improvement over one that asserts the wrong one.
Why it matters beyond wording
With catch_up: false a skipped firing is lost rather than deferred, so the count in ob status is missed ingestion, not delayed ingestion. An operator who believes the cause is their own deploy cadence will change something that cannot help, while the real cause — schedule density — goes unexamined.
This is adjacent to #133 but not the same: that is about deploys proceeding during long jobs, this is about a job-versus-job collision being reported as an application operation.
- 主要语言
- Go
- 星标
- 4
- 派生
- 0
- 平均合并
- 2 小时 46 分钟
- 30 天内合并 PR
- 38
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
labstack/onebox 的其他 Issue
-
bug
难度 1/5 1 小时以内 新手友好度 80/100
维护者通常 1 天内回复
-
bug
难度 1/5 1 小时以内 新手友好度 85/100
维护者通常 1 天内回复
-
bug
难度 1/5 1 小时以内 新手友好度 85/100
维护者通常 1 天内回复
-
bug
难度 3/5 1-2 天 新手友好度 65/100
维护者通常 1 天内回复
-
难度 4/5 3-5 天 新手友好度 55/100
维护者通常 1 天内回复
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 84/100
维护者通常 1 天内回复
-
[Bug] Extraction-mode autogen returns pod templates in random order for map-keyed CRDs, so deny and report messages change between runs可能已有人在做 @ManojLamani 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 75/100
kyverno/kyverno#17980 · 1 个 reaction ·
维护者通常 1 天内回复
-
raised-by:reviewer
难度 1/5 1-3 小时 新手友好度 84/100
medici-finance/assay#2681 ·
维护者通常 1 天内回复
-
bug triaged
难度 2/5 1-3 小时 新手友好度 78/100
fullsend-ai/fullsend#8353 · 4 条评论 · 1 个 reaction ·
维护者通常 1 天内回复
-
duplication
难度 2/5 1-3 小时 新手友好度 76/100
openvibely/openvibely#1459 ·
维护者通常 2 天内回复