Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Make local E2E runs deterministic with zero skipped tests

Aperta
  • Qualcosa non va?
#13 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
35/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
docker, go, helm, java, kubernetes, python, shell
Ambito
devops, documentation, infrastructure, testing-qa

Direzione di ricerca

Read integration-tests/e2e/run.sh and integration-tests/e2e/lib.sh first to trace E2E_FAIL and E2E_SKIP handling, then inspect Tiers 5, 8, 9, 12, and 14 plus integration-tests/AGENTS.md. Run the documented local harness and define preflight and strict-mode behavior so required checks are actionable, strict runs finish with zero failures and skips, and default skip behavior remains portable.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Problem

The end-to-end harness currently treats skipped tests as a successful run. integration-tests/e2e/lib.sh tracks E2E_SKIP, but integration-tests/e2e/run.sh exits non-zero only when E2E_FAIL is non-zero.

This can hide incomplete local coverage caused by missing prerequisites, unsupported cluster topology, or state leaking between tiers. For example, a full local run skipped Tier 14's containerd rollback scenario because Tier 8 had left a Brewlet runtime stanza in the Docker Desktop worker's /etc/containerd/config.toml. After removing that stale stanza, Tier 14 completed with 19 passed, 0 failed, and 0 skipped.

Proposed work

  • Add an opt-in strict mode such as --require-all that exits non-zero when any test is skipped.
  • Add a preflight that reports missing requirements before starting: Go, JDK 21+, Python 3, Maven, Docker, kubectl, Helm, OpenSSL, network access, a reachable cluster, and a schedulable local containerd node.
  • Isolate node-mutating tiers, preferably using fresh disposable clusters aligned with the CI matrix, or ensure each tier snapshots and restores node state.
  • Update Tier 8, Tier 9, and Tier 12 cleanup so containerd configuration and other node-side changes cannot affect Tier 14 or subsequent runs.
  • Make Tier 5's host-bound webhook connectivity portable rather than relying exclusively on host.docker.internal, or provide a deterministic supported local configuration.
  • Document the strict local invocation and required environment in integration-tests/AGENTS.md.

Acceptance criteria

  • integration-tests/e2e/run.sh --reset --require-all succeeds only when the summary reports zero failures and zero skips.
  • Missing prerequisites fail during preflight with actionable messages.
  • Repeated full local runs produce no state-related skips.
  • Tier 14's rollback assertions execute after Tiers 8, 9, and 12 in a full local run.
  • The default non-strict mode retains its current portable skip behavior.
  • CI and local-run documentation explain the supported cluster setup and strict mode.
Lingua principale
Go
Stelle
0
Fork
0
Merge medio
11h 48m
PR unite (30g)
99

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di microsoft/brewlet

Tutte le issue di microsoft/brewlet

Issue simili

Altre issue su Go

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.