CI: a container job that starts seconds after the dev image is pushed cannot pull it

Abierto
#2,385 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
35/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
docker, github-actions
Área
ci-cd

Línea de trabajo

Start with ci.yml and the build dev/ci image job, then read tools/ci_tier_check.sh and its self-test. Compare the proposed pull check, attestation handling, and longer retry against the failure logs; the issue requires recording one choice and its reason. Done when the chosen check is logged on pull request, merge group, and dispatched runs, the tier check passes, and the requested Actions API count is added here.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

area:ci found-by:ci kind:flake

Seen on a dispatched full lane, run 44020220329 (ci.yml, lanes=all, on a branch at 5092ba79), 2026-11-17.

What happened

The job build dev/ci image pushed ghcr.io/inauguralsystems/eigenscript-dev:ci-5092ba79… and ended at 04:55:29Z. Four container jobs that need it started between 04:55:31Z and 04:55:40Z, and each failed in its "Initialize containers" step before any step of ours ran:

docker pull ghcr.io/inauguralsystems/eigenscript-dev:ci-5092ba793fbc4761cd5e1959e349d79ee086f65e
Error response from daemon: failed to resolve reference "ghcr.io/…/eigenscript-dev:ci-5092ba79…"
##[warning]Docker pull failed with exit code 1, back off 9.658 seconds before retry.
failed to copy: httpReadSeeker: failed open: could not fetch content descriptor
  sha256:6a8f0165885b4a4dedec34bb6adf8d127f561ec2eb626133159ce8d02cc568e5 (application/vnd.in-toto+json) from remote: not found
##[warning]Docker pull failed with exit code 1, back off 8.734 seconds before retry.
Error response from daemon: failed to resolve reference "ghcr.io/…/eigenscript-dev:ci-5092ba79…"
##[error]Docker pull failed with exit code 1

The failed jobs: db extension (postgres service), db extension suite (postgres service), extensions / server (http+net+gfx) and ancillary checks, replay differential (same-binary tape fidelity); and extensions (http+net+gfx suite; embed/lsp/jit-smoke) after them, as their aggregate. The container jobs that started later in the same run pulled the same tag and passed. The change under test was documents only.

What it looks like

The tag resolves a few seconds after the push returns, and the blob the second attempt could not fetch is the attestation that docker/build-push-action adds to the image index. The runner tries three times in about twenty seconds and gives up. So a job that is scheduled at once after the image job can lose a race with the registry.

Done when

  • Decided and recorded here: the image job proves the tag can be pulled before it ends (a pull by the pushed digest with a bounded retry), or the image is pushed without the attestation manifest, or the container jobs get a longer retry. With the reason for the one chosen.
  • The image job's log shows the check that was chosen, on a pull_request run, a merge_group run and a dispatched run.
  • tools/ci_tier_check.sh passes with the change, and its self-test covers the new step if it adds one.
  • A count from the Actions API, in this issue: of the last 200 runs of ci.yml, how many had a container job fail in "Initialize containers" with this message. That says how often the race is lost today.

— Claude DevBox

Lenguaje dominante
C
Estrellas
3
Forks
7
Merge medio
3 h 28 min
PR fusionados (30 d)
270

Preparar el entorno

Abrir en Codespaces

Inicia el contenedor de desarrollo del proyecto en tu navegador, con tu propia cuenta de GitHub.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de InauguralSystems/EigenScript

Todos los issues de InauguralSystems/EigenScript

Issues similares

Más issues de C

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.