Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Pre-push redaction + lint pipeline — make the repo site-agnostic by construction

Aperta
#33 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
35/100
Tipo di issue
Funzionalità
Chiarezza
Da chiarire
Stato di attività
Tranquilla
Stack tecnologico
git, github-actions, python

Direzione di ricerca

Inizia leggendo .github/workflows/ci.yml e tests/test_provenance.py per comprendere il gate esistente di controllo dei contenuti. Quindi mappa le scansioni pre-push e CI proposte sui file tracciati, sui file non tracciati non ignorati e sui messaggi dei commit inviati, incluse le opzioni di sincronizzazione dell'allowlist e dei pattern dei secret. Il lavoro è completato quando il gate fallisce in modo sicuro senza passare silenziosamente quando la chiave richiesta o l'insieme di pattern non è disponibile.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Why

This repo is public, so every commit is world-readable permanently. We keep a manual redaction
recipe in a local, gitignored manifest and run it by hand "before any commit that touches notes/,
docs/ or the repo root".

That discipline just failed a measurable test. A manual pass on 2026-08-04 (commits d2810ba,
ab68070, de281fb) found, in the working tree:

category hits where
An end-of-life interpreter present on the site workstation, named by version 5 2 notes, 1 plan, 1 spec
The site IFO environment-variable value, stated outright 7 3 notes, 1 plan, 1 spec
The site channel-namespace prefix, used as the channel prefix for synthetic twin channels 133 17 files: doc examples, tutorial, demo configs, tests, one experiment
Site filesystem paths (foton bank, live-site chans dir, noise-budget ref) 5 1 note
A named person's personal home path 1 1 note

Two things this makes obvious:

  1. A recipe that depends on someone remembering to run it does not get run. Most of what was
    found had already been pushed — the channel prefix had been public on main since June/July.
    Redacting the tip does not unpublish it.
  2. File scanning is not sufficient. The file scan was already green when a scan of the commit
    messages
    in the unpushed range caught two further leaks — including, embarrassingly, the
    redaction commit whose own subject line named the string it was redacting. Messages are published
    verbatim on push. Both were fixed while still local.

Proposal

A pre-push hook plus a CI leg that runs one pattern set over:

  • tracked files;
  • untracked-but-not-ignored files (the set that would be committed);
  • commit messages in the range being pushed.

Fail the push/build on any match. Ship an allowlist for the known false-positive classes, which are
real and varied — filter coefficients whose digits coincide with a port number, unrelated dependency
version pins, base64 image payloads inside frozen Quarto caches, single-letter-plus-digit
state-space matrix variables, lettered document finding labels, and a smoothness class (C¹) in a
commit message about ramp shape. A gate that cries wolf on those gets disabled within a week, so the
allowlist is part of the deliverable, not an afterthought.

.github/workflows/ci.yml already has the right precedent in the provenance gate
(tests/test_provenance.py) — a content check that fails the build. This can follow that shape.

The hard part: synchronising the secret

The pattern set is itself sensitive. To match on a string you generally need the string, so
committing a scanner that greps for these values into a public repo republishes them — the scanner
becomes the leak. That is the actual blocker, and it is why the current recipe is a local file rather
than a CI job.

Options, none free:

  1. Store salted hashes of the literals, not the literals. Tokenise each candidate line and
    compare digests. The plaintext never lands in the repo. Costs substring/regex matching — you can
    only match whole tokens you thought to enumerate.
  2. Split the set: structural patterns committed, literals local. What we do today. The committed
    half (IFO=<letter><digit>, <alias>:<port>, interpreter-version shapes) is genuinely useful and
    leaks nothing; the literal half stays gitignored. Weakness: CI can only ever enforce the
    structural half.
  3. Private sidecar repo or submodule holding the literal list, with CI given a read token.
    Clean separation; adds a second repo to administer and a token to rotate.
  4. Encrypted in-repo blob (git-crypt / age / sops), decrypted by an Actions secret in CI
    and by a per-collaborator key locally. Single repo, real coverage, but now key distribution and
    revocation are ours to run.

Whichever we pick, the gate must fail closed. The dangerous failure mode is a new collaborator
with no key getting a scanner that silently passes — worse than no gate at all, because it manufactures
confidence. A hard failure with a clear "you need the key, ask X" message is correct.

Bonus: this makes the repo site-agnostic by construction

The same gate that blocks the site IFO value blocks any hardcoded site channel name. That is
already a stated design requirement — the CDS backend spec says Component 1 must not hard-code a
single site channel name or the site IFO value so that other hardware needs only a new profile
(docs/superpowers/specs/2026-08-03-cds-hardware-backend-design.md, and the site-profile work in
#26). Right now that requirement is enforced by review, and review had let 133 instances through.

It also gives contributors something concrete to reach for. The 2026-08-04 pass established a
placeholder vocabulary for prose — <host>, <remote-path>, <conda-root>, <IFO> for the site
prefix and <MDL> for the subsystem/front-end model element. <MDL> matters beyond redaction: it
stops channel templates in the notes from hardcoding the suspension paradigm, so they read as
"any model" rather than "this one subsystem".

One caveat worth stating so nobody oversells this: for a project whose site is named in the repo,
placeholders reduce greppability, not inferability. Anyone familiar with LIGO CDS can infer the
prefix. The genuine wins are that the values stop being machine-harvestable from a public repo, and
that the code stops depending on them.

Appendix: history rewrite — proposal only, for a maintainer decision

Recorded because the redaction above fixes only the tip. Not executed, and not something to do
unilaterally.

Affected (SHAs and paths are already public; the strings are what must not be restated):

  • origin/main @ f13c293 and earlier — site channel prefix, across docs/examples/01–06,
    docs/tutorial/closing-the-loop.qmd, src/system_ident/configs/*.yml, five test modules,
    experiments/prior_bakeoff/refinement_sweep.py, and notes/40m-sos-campaign-handoff-2026-07.md
    (which additionally carries the site paths and the personal home path).
  • origin/feat/cds-hardware-backend @ d9d3432 — interpreter version ×3 and site IFO value ×5, in
    docs/superpowers/plans/2026-08-03-cds-hardware-backend.md,
    docs/superpowers/specs/2026-08-03-cds-hardware-backend-design.md, and
    notes/cds-hardware-bringup-2026-08.md.

Shape, if approved:

git filter-repo --replace-text <replacements-file>

The replacements file is generated locally from the gitignored manifest and never committed — the
same paradox as above applies to the rewrite tooling.

Costs, stated honestly:

  • Every collaborator must re-clone; stale clones reintroduce the old objects on their next push.
  • Open PRs break and need recreating.
  • Existing forks keep the old objects, and we cannot reach into them.
  • GitHub keeps unreachable commits fetchable by SHA unless Support is asked to purge them, and
    the SHAs are in this issue and in the reflogs of anyone who cloned.
  • Third-party mirrors, the rendered Pages site, and search-engine caches are all outside our control.

So a rewrite lowers future discoverability but does not guarantee retraction, and it imposes real
coordination cost on everyone else.

Recommendation: accept the historical exposure, redact going forward, and land the gate in this
issue so the exposure stops growing. Nothing found here is a credential — no key, token or password
was ever committed. The categories are a service endpoint, an interpreter version, and a naming
prefix. Worth stopping; probably not worth a shared-history rewrite. Reassess if a future finding is
an actual secret.

Lingua principale
Python
Stelle
0
Fork
1
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di CaltechExperimentalGravity/system_ident

Tutte le issue di CaltechExperimentalGravity/system_ident

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.