CI: a container job that starts seconds after the dev image is pushed cannot pull it

未关闭
#2,385 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
4/5
预计耗时
3-5 天
新手友好度
35/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
活跃
技术栈
docker, github-actions
领域
ci-cd

调研方向

Start with ci.yml and the build dev/ci image job, then read tools/ci_tier_check.sh and its self-test. Compare the proposed pull check, attestation handling, and longer retry against the failure logs; the issue requires recording one choice and its reason. Done when the chosen check is logged on pull request, merge group, and dispatched runs, the tier check passes, and the requested Actions API count is added here.

由索引模型根据 Issue 内容生成。

描述

area:ci found-by:ci kind:flake

Seen on a dispatched full lane, run 44020220329 (ci.yml, lanes=all, on a branch at 5092ba79), 2026-11-17.

What happened

The job build dev/ci image pushed ghcr.io/inauguralsystems/eigenscript-dev:ci-5092ba79… and ended at 04:55:29Z. Four container jobs that need it started between 04:55:31Z and 04:55:40Z, and each failed in its "Initialize containers" step before any step of ours ran:

docker pull ghcr.io/inauguralsystems/eigenscript-dev:ci-5092ba793fbc4761cd5e1959e349d79ee086f65e
Error response from daemon: failed to resolve reference "ghcr.io/…/eigenscript-dev:ci-5092ba79…"
##[warning]Docker pull failed with exit code 1, back off 9.658 seconds before retry.
failed to copy: httpReadSeeker: failed open: could not fetch content descriptor
  sha256:6a8f0165885b4a4dedec34bb6adf8d127f561ec2eb626133159ce8d02cc568e5 (application/vnd.in-toto+json) from remote: not found
##[warning]Docker pull failed with exit code 1, back off 8.734 seconds before retry.
Error response from daemon: failed to resolve reference "ghcr.io/…/eigenscript-dev:ci-5092ba79…"
##[error]Docker pull failed with exit code 1

The failed jobs: db extension (postgres service), db extension suite (postgres service), extensions / server (http+net+gfx) and ancillary checks, replay differential (same-binary tape fidelity); and extensions (http+net+gfx suite; embed/lsp/jit-smoke) after them, as their aggregate. The container jobs that started later in the same run pulled the same tag and passed. The change under test was documents only.

What it looks like

The tag resolves a few seconds after the push returns, and the blob the second attempt could not fetch is the attestation that docker/build-push-action adds to the image index. The runner tries three times in about twenty seconds and gives up. So a job that is scheduled at once after the image job can lose a race with the registry.

Done when

  • Decided and recorded here: the image job proves the tag can be pulled before it ends (a pull by the pushed digest with a bounded retry), or the image is pushed without the attestation manifest, or the container jobs get a longer retry. With the reason for the one chosen.
  • The image job's log shows the check that was chosen, on a pull_request run, a merge_group run and a dispatched run.
  • tools/ci_tier_check.sh passes with the change, and its self-test covers the new step if it adds one.
  • A count from the Actions API, in this issue: of the last 200 runs of ci.yml, how many had a container job fail in "Initialize containers" with this message. That says how often the race is lost today.

— Claude DevBox

主要语言
C
星标
3
派生
7
平均合并
3 小时 28 分钟
30 天内合并 PR
270

环境准备

在 Codespaces 中打开

在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

InauguralSystems/EigenScript 的其他 Issue

查看 InauguralSystems/EigenScript 的全部 Issue

相似的 Issue

更多 C Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。