Release gating: a gate.yml in the canary tests the candidate before v0 moves

Aperta
  • Qualcosa non va?
#135 4 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
45/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Tranquilla
Stack tecnologico
github-actions, shell
Ambito
ci-cd, devops, release

Direzione di ricerca

Inizia leggendo il processo di release di questo repository e i workflow di canary in QuantEcon/test-actions-lecture-intro, in particolare ci.yml e publish.yml. Esamina i riferimenti corrispondenti in build-jupyter-cache/action.yml alle righe 149, 164, 174 e 184. Il lavoro è completato quando v0-next prepara il candidato, il risultato del canary impedisce di far avanzare v0, rollback e responsabilità sono documentati e il gate e gli avvisi umani sono stati deliberatamente testati in autonomia.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

high-priority infrastructure

Decision from the consumer side: a red canary should block a release. The lecture family wants quantecon/actions verified green — within the bounds of current testing — before a release is published, and is sequencing its move to floating @v0 behind that. Consumer-side context in QuantEcon/workspace-lectures#33; this issue is the implementation.

This is stage 4 of #100, plus the thing #100 did not specify: how a gate is supposed to work when the canary is pinned to the very tag the release moves.

The blocker: the canary cannot gate a release as built

Every one of the canary's action references is @v0:

Ref Count in test-actions-lecture-intro
build-lectures@v0 6
build-jupyter-cache@v0 1
restore-jupyter-cache@v0 1
preview-netlify@v0 1
publish-gh-pages@v0 1

v0 is what a release moves. So the canary always exercises the previous release, never the candidate — it is a post-release regression detector, and no amount of scheduling changes that. "Green before we release" is not expressible against it today.

Two further facts shape any fix. GitHub does not allow expressions in uses:, so a ref can never be parameterised. And build-jupyter-cache reaches its siblings at a hardcoded @v0 — setup-environment@v0 at action.yml:149 and build-lectures@v0 at :164, :174, :184.

Proposal: a v0-next staging tag

Add a second floating tag that always points at the release candidate, and give the canary a workflow set pinned to it. The release then becomes:

  1. move v0-next to the candidate commit;
  2. dispatch the canary's v0-next workflows;
  3. green is a precondition for step 4 — red stops the release;
  4. move v0 to the same commit.

This works precisely because it does not need parameterised refs: @v0-next is as static as @v0. It costs a duplicate workflow set in the canary, which is the price of the uses: constraint rather than a design smell.

What it gates, and what it cannot

Gated — every action the canary calls directly, which is the whole publish path and the highest-consequence surface:

publish-gh-pages, preview-netlify, build-lectures, restore-jupyter-cache, setup-environment.

Not gated — the build-jupyter-cache sibling chain. A build-jupyter-cache@v0-next run still executes setup-environment@v0 and build-lectures@v0, i.e. the previous release's siblings. The candidate's cache action is tested against stale siblings, so the pairing that consumers will actually run remains untested until after v0 moves. That is the structural limit named in #100 and this proposal does not remove it; it narrows the ungated surface to one action's internal chain.

Worth stating plainly so the gate is not over-trusted: "100% before release" will mean "everything except the build-jupyter-cache sibling chain", unless that chain is addressed separately (see below).

Options for the residual, not recommended unilaterally
  • Rewrite sibling refs at release time. A release step rewrites @v0 → the candidate SHA in build-jupyter-cache/action.yml on the tagged tree. Fully self-consistent and testable, but the tagged tree then differs from main, which is a real cost to reasoning about the repo.
  • Accept it, and lean on cadence. The cache path already has the longest post-release lead of any consumer path — the canary's weekly build runs Sunday 03:00 UTC against lecture repos' Monday 02:00/03:00 UTC. Combined with rollback, exposure is bounded to roughly a day.

Work

Gate mechanism

  • Introduce the v0-next tag and document its meaning alongside v0
  • Add a v0-next workflow set to QuantEcon/test-actions-lecture-intro
  • Add a release step that moves v0-next, dispatches the canary, and requires green before v0 moves
  • Document the rollback: v0 is movable, so recovery from a bad release is git tag -f v0 <previous>. Write it down before it is needed, with who is expected to act

Close the ungated consumer paths (these matter whether or not the gate lands, since they are also the post-release safety net)

  • Schedule the canary's ci.yml — the preview path currently runs only when a human opens a PR in the canary (last: 2026-08-05, before that 2026-07-15)
  • Schedule the canary's publish.yml, and give it a workflow_dispatch — it currently has neither, so it cannot even be run by hand
  • Keep any canary schedule strictly ahead of consumers' Monday 02:00/03:00 UTC cache builds

Prove the gate is real

A gate that has never been exercised is not a gate, and this repo has precedent on both halves: the container smoke tests could not fail for three independent reasons and still reported green (#108), and the canary failed eight consecutive weeks (2026-02-22 → 2026-04-12) filing zero issues (#83).

  • Self-test the gate — stage a deliberately broken candidate, confirm the canary goes red and the release is actually stopped, then revert
  • Confirm a red canary reaches a human, separately from the above. The eight silent weeks were failures that alerted nobody

Open questions

Question Why it matters
Is the release blocked automatically, or by convention in the runbook? There is no release automation here today — v0 is moved by hand — so "blocked" may initially mean a documented step rather than an enforced one
Does v0-next also get a scheduled canary run, or only on release? Only-on-release is cheaper; scheduled would catch drift on main between releases
Is the build-jupyter-cache residual acceptable? It decides whether "100%" is qualified or whether the ref-rewrite option is worth its cost

Related

  • #100 — testing; this is its stage 4 and the gating design it left open
  • #83 — the two-month silent alerting failure behind the "prove it reaches a human" item
  • #108 — the smoke tests that could not fail; precedent for self-testing the gate
  • QuantEcon/workspace-lectures#33 — the consumer-side plan, blocked on this
  • QuantEcon/workspace-lectures#31 — the lecture-family CI migration this unblocks
Lingua principale
Shell
Stelle
0
Fork
1
Merge medio
20h 3m
PR unite (30g)
48

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di QuantEcon/actions

Tutte le issue di QuantEcon/actions

Issue simili

Altre issue su Shell/Bash

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.