Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Build recipes that exceed the 6h runner limit (e.g. envoy) via a Bazel remote cache

Aperta
  • Qualcosa non va?
#13,958 4 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
35/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Tranquilla
Stack tecnologico
github-actions, rust
Ambito
build-system, ci-cd, infrastructure

Direzione di ricerca

Start by reviewing the build workflow and the remote-cache recipe proposed in #13957, then verify how the Envoy build currently passes Bazel flags. Done means a persistent cache is reachable securely, sequential jobs share it, and the build can resume across windows to produce the bottle.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Problem

github.com/envoyproxy/envoy cannot be bottled on the standard GitHub-hosted
runners: a from-scratch Bazel build exceeds the 6 h job time limit. This is
not a recipe defect — every recipe-level lever has been exhausted:

Lever tried Result
--jobs = HOST_CPUS * 0.5 x86-64 times out at 6 h
--jobs = HOST_CPUS * 1.0 ARM64 OOM-killed
--jobs = HOST_CPUS * 0.66 both fail (x64 timeout, ARM64 flaky Rust genrule)
--fission=no, -Wno-error, --disable-new-dtags, rustc wrapper needed to compile at all, but do not reduce wall-clock below 6 h

A single job cannot finish, and the recipe cannot change the runner's time limit.

Key insight (empirically verified)

Bazel uploads each action's result to a remote cache as soon as that action
completes — not at the end of the build. Consequently a fresh, ephemeral
runner pointed at the same remote cache hits the cache for every action a
previous run already completed, and only executes what remains. Splitting one

6 h build into several sequential ≤6 h windows becomes viable: window K
uploads the actions it finishes before timing out; window K+1 starts from a
clean checkout, hits the cache for those actions, and continues.

Proof

Reference cache: bazel-remote, built
pure-Go, CGO=0. Test project: 6 × sleep 4 genrules + an aggregator.

PHASE 1  COLD  (cache empty)        -> 7 processes executed        10 s
         bazel clean --expunge      -> wipes the ENTIRE output_base (= a brand-new runner)
PHASE 2  WARM  (same remote_cache)  -> 7 remote cache hit, 0 exec   2 s
         cache store survived the --expunge: 28 files / 112 K

--expunge deletes the whole local build state, which is exactly what an
ephemeral runner starts from. The warm run reconstructed the action graph and
resumed entirely from the cache — no re-execution. This is the multi-window
mechanism, demonstrated end to end.

Proposal

  1. Stand up a persistent Bazel remote cache reachable from the build workflow —
    e.g. bazel-remote (recipe proposed in #13957), backed by disk or S3, with
    GC/quota. One small always-on host, or an S3-proxy cache, suffices.
  2. Add to the envoy build the flags: 
    --remote_cache=$CACHE_URL --remote_upload_local_results=true
    The cache URL / credentials are a workflow-level secret, not part of the
    public recipe (a public recipe must not carry secrets). This is why the
    change belongs at the workflow layer, not in package.yml.
  3. Run the build as N sequential jobs (a small needs:-ordered chain), each
    with the same --remote_cache. Each job may time out; the next resumes from
    the cache. After ≤N windows the final job produces a fully-populated tree and
    the bottle is packaged.

Alternatives considered

  • --disk_cache on a persistent runner volume — the local analogue; works
    only if the same runner volume survives between windows. Ephemeral hosted
    runners don't keep the volume, so a remote cache is the portable answer.
    (For a self-hosted runner with a persistent workspace, --disk_cache alone
    is enough and needs no server.)
  • A self-hosted runner without the 6 h cap — solves it directly but adds
    standing infra + trust surface; the remote-cache approach keeps the hosted
    runners.
  • RBE (BuildBuddy / NativeLink) — remote execution, not just caching;
    bigger lift, out of scope for "just get envoy to bottle".

Ask

Is pkgxdev open to (a) a persistent remote cache in the build infra and (b) a
chained multi-window job for the few recipes that exceed 6 h? If so, envoy
(otherwise code-complete) can land. Happy to wire the workflow side.

Lingua principale
C
Stelle
199
Fork
141
Merge medio
6h 50m
PR unite (30g)
119

Preparare l'ambiente

Apri in Codespaces

Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.

  • Nessun Dockerfile né file Docker Compose
  • Ha un modello di pull request
  • Nessuna guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di pkgxdev/pantry

Tutte le issue di pkgxdev/pantry

Issue simili

Altre issue su C

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.