Define reproducible archive metadata for release and Nightly packaging

Aperta
  • Qualcosa non va?
#1,023 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
38/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python, rust
Ambito
build-system, release

Direzione di ricerca

Start with the archive writer at tools/ci/package.py:299 and trace the package_targets serialization path for tar.gz and zip output. Review the existing deterministic stage and gzip-clock fixtures, then identify where timestamps, ownership, entry order, and permissions are set; the issue requires agreeing on an epoch and metadata contract with @LunaStev before implementation. Done means repeated runs with identical source, toolchain, and epoch produce byte-identical archives while permissions, extracted-package smoke behavior, and identity/checksum/manifest binding remain correct.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

ci discussion enhancement help wanted

Current problem after the Rust tooling migration

Keep this issue open for design. PR #1071 retired tools/ci/package.py; the current writer is wave_tools::package::archive in tools/src/package.rs. Moving the implementation to Rust did not select a reproducible archive-metadata contract.

The Rust tar.gz branch calls tar::Builder::append_dir_all on the staging tree and retains file metadata such as mtime, UID/GID and permissions. Entry ordering also needs an explicit contract. The gzip header itself currently uses mtime 0; the earlier Python-specific claim about ambient gzip creation time does not describe this Rust writer.

Reproduction on the merged implementation

On November 2, exercised the actual Rust archive function from the tree merged as e40018fcfa1a6ab99acd24f7d729aaee3d59f5f7. A temporary stage contained one unchanged payload file with fixed bytes. Two calls changed only the payload mtime, from 1600000000 to 1700000000; the same archive root name was used in both runs. No production artifact was published.

// With a temporary stage containing fixed payload bytes:
file.set_times(std::fs::FileTimes::new().set_modified(
    std::time::UNIX_EPOCH + std::time::Duration::from_secs(mtime),
))?;
wave_tools::package::archive(&stage, &output, "wave-probe")?;

Observed SHA-256 values:

tar.gz, first:  8a511063599284b7459dcf5a5b9db6832aadc190fd7d74bccc1500f0460ad9fc
tar.gz, second: 3f8221c5027bd60516c1a142cd4d9f624afa987ecaed4a79e73c90a2da1718f5
zip, both:     5e8a3f8b433350ab7394715ce222a6692a52375bee74374c51f1afe42583ae0b

Inspection confirmed the two tar entries retain their distinct input mtimes while both gzip headers have mtime 0. The ZIP outputs stayed identical in this mtime-only probe; this is not evidence of nondeterministic ZIP timestamps. ZIP permission/order policy still belongs in the design, without assuming the old Python behavior survives.

Scope and design discussion

Choose the epoch, timestamps, ownership, entry order and permission-normalization rules for tar.gz and ZIP. Preserve executable permissions, runtime layout, extracted-package behavior and identity/checksum binding to the exact generated bytes. Do not reintroduce the retired Python writer.

Please mention @LunaStev to discuss the design before starting implementation.

Completion criteria

  • Identical source/toolchain/approved epoch produces byte-identical archives in isolated runs.
  • Tests vary ambient file timestamps and ownership/order inputs where supported.
  • Permissions and extracted-package smoke checks remain correct.
  • Checksums, metadata and manifests identify the exact resulting archive bytes.
Lingua principale
Rust
Stelle
56
Fork
21
Merge medio
23h 20m
PR unite (30g)
45

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di wavefnd/Wave

Tutte le issue di wavefnd/Wave

Issue simili

Altre issue su Rust

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.