CI: a container job that starts seconds after the dev image is pushed cannot pull it

Ouverte
#2,385 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 1 jour

Personne n'a encore pris cette issue.

Évaluation

Difficulté
4/5
Temps estimé
3-5 jours
Accessibilité débutants
35/100
Type d'issue
Bug
Clarté
Clairement spécifiée
Activité
Active
Stack technique
docker, github-actions
Domaine
ci-cd

Piste de recherche

Start with ci.yml and the build dev/ci image job, then read tools/ci_tier_check.sh and its self-test. Compare the proposed pull check, attestation handling, and longer retry against the failure logs; the issue requires recording one choice and its reason. Done when the chosen check is logged on pull request, merge group, and dispatched runs, the tier check passes, and the requested Actions API count is added here.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

area:ci found-by:ci kind:flake

Seen on a dispatched full lane, run 44020220329 (ci.yml, lanes=all, on a branch at 5092ba79), 2026-11-17.

What happened

The job build dev/ci image pushed ghcr.io/inauguralsystems/eigenscript-dev:ci-5092ba79… and ended at 04:55:29Z. Four container jobs that need it started between 04:55:31Z and 04:55:40Z, and each failed in its "Initialize containers" step before any step of ours ran:

docker pull ghcr.io/inauguralsystems/eigenscript-dev:ci-5092ba793fbc4761cd5e1959e349d79ee086f65e
Error response from daemon: failed to resolve reference "ghcr.io/…/eigenscript-dev:ci-5092ba79…"
##[warning]Docker pull failed with exit code 1, back off 9.658 seconds before retry.
failed to copy: httpReadSeeker: failed open: could not fetch content descriptor
  sha256:6a8f0165885b4a4dedec34bb6adf8d127f561ec2eb626133159ce8d02cc568e5 (application/vnd.in-toto+json) from remote: not found
##[warning]Docker pull failed with exit code 1, back off 8.734 seconds before retry.
Error response from daemon: failed to resolve reference "ghcr.io/…/eigenscript-dev:ci-5092ba79…"
##[error]Docker pull failed with exit code 1

The failed jobs: db extension (postgres service), db extension suite (postgres service), extensions / server (http+net+gfx) and ancillary checks, replay differential (same-binary tape fidelity); and extensions (http+net+gfx suite; embed/lsp/jit-smoke) after them, as their aggregate. The container jobs that started later in the same run pulled the same tag and passed. The change under test was documents only.

What it looks like

The tag resolves a few seconds after the push returns, and the blob the second attempt could not fetch is the attestation that docker/build-push-action adds to the image index. The runner tries three times in about twenty seconds and gives up. So a job that is scheduled at once after the image job can lose a race with the registry.

Done when

  • Decided and recorded here: the image job proves the tag can be pulled before it ends (a pull by the pushed digest with a bounded retry), or the image is pushed without the attestation manifest, or the container jobs get a longer retry. With the reason for the one chosen.
  • The image job's log shows the check that was chosen, on a pull_request run, a merge_group run and a dispatched run.
  • tools/ci_tier_check.sh passes with the change, and its self-test covers the new step if it adds one.
  • A count from the Actions API, in this issue: of the last 200 runs of ci.yml, how many had a container job fail in "Initialize containers" with this message. That says how often the race is lost today.

— Claude DevBox

Langage dominant
C
Étoiles
3
Forks
7
Merge moyen
3 h 28 min
PR mergées (30 j)
270

Préparer son environnement

Ouvrir dans Codespaces

Lance le conteneur de développement du projet dans votre navigateur, avec votre propre compte GitHub.

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de InauguralSystems/EigenScript

Toutes les issues de InauguralSystems/EigenScript

Issues similaires

Plus d'issues C

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.