Pre-push redaction + lint pipeline — make the repo site-agnostic by construction
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Idoneità per principianti
- 35/100
Direzione di ricerca
Inizia leggendo .github/workflows/ci.yml e tests/test_provenance.py per comprendere il gate esistente di controllo dei contenuti. Quindi mappa le scansioni pre-push e CI proposte sui file tracciati, sui file non tracciati non ignorati e sui messaggi dei commit inviati, incluse le opzioni di sincronizzazione dell'allowlist e dei pattern dei secret. Il lavoro è completato quando il gate fallisce in modo sicuro senza passare silenziosamente quando la chiave richiesta o l'insieme di pattern non è disponibile.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Why
This repo is public, so every commit is world-readable permanently. We keep a manual redaction
recipe in a local, gitignored manifest and run it by hand "before any commit that touches notes/,
docs/ or the repo root".
That discipline just failed a measurable test. A manual pass on 2026-08-04 (commits d2810ba,
ab68070, de281fb) found, in the working tree:
| category | hits | where |
|---|---|---|
| An end-of-life interpreter present on the site workstation, named by version | 5 | 2 notes, 1 plan, 1 spec |
| The site IFO environment-variable value, stated outright | 7 | 3 notes, 1 plan, 1 spec |
| The site channel-namespace prefix, used as the channel prefix for synthetic twin channels | 133 | 17 files: doc examples, tutorial, demo configs, tests, one experiment |
| Site filesystem paths (foton bank, live-site chans dir, noise-budget ref) | 5 | 1 note |
| A named person's personal home path | 1 | 1 note |
Two things this makes obvious:
- A recipe that depends on someone remembering to run it does not get run. Most of what was
found had already been pushed — the channel prefix had been public onmainsince June/July.
Redacting the tip does not unpublish it. - File scanning is not sufficient. The file scan was already green when a scan of the commit
messages in the unpushed range caught two further leaks — including, embarrassingly, the
redaction commit whose own subject line named the string it was redacting. Messages are published
verbatim on push. Both were fixed while still local.
Proposal
A pre-push hook plus a CI leg that runs one pattern set over:
- tracked files;
- untracked-but-not-ignored files (the set that would be committed);
- commit messages in the range being pushed.
Fail the push/build on any match. Ship an allowlist for the known false-positive classes, which are
real and varied — filter coefficients whose digits coincide with a port number, unrelated dependency
version pins, base64 image payloads inside frozen Quarto caches, single-letter-plus-digit
state-space matrix variables, lettered document finding labels, and a smoothness class (C¹) in a
commit message about ramp shape. A gate that cries wolf on those gets disabled within a week, so the
allowlist is part of the deliverable, not an afterthought.
.github/workflows/ci.yml already has the right precedent in the provenance gate
(tests/test_provenance.py) — a content check that fails the build. This can follow that shape.
The hard part: synchronising the secret
The pattern set is itself sensitive. To match on a string you generally need the string, so
committing a scanner that greps for these values into a public repo republishes them — the scanner
becomes the leak. That is the actual blocker, and it is why the current recipe is a local file rather
than a CI job.
Options, none free:
- Store salted hashes of the literals, not the literals. Tokenise each candidate line and
compare digests. The plaintext never lands in the repo. Costs substring/regex matching — you can
only match whole tokens you thought to enumerate. - Split the set: structural patterns committed, literals local. What we do today. The committed
half (IFO=<letter><digit>,<alias>:<port>, interpreter-version shapes) is genuinely useful and
leaks nothing; the literal half stays gitignored. Weakness: CI can only ever enforce the
structural half. - Private sidecar repo or submodule holding the literal list, with CI given a read token.
Clean separation; adds a second repo to administer and a token to rotate. - Encrypted in-repo blob (
git-crypt/age/sops), decrypted by an Actions secret in CI
and by a per-collaborator key locally. Single repo, real coverage, but now key distribution and
revocation are ours to run.
Whichever we pick, the gate must fail closed. The dangerous failure mode is a new collaborator
with no key getting a scanner that silently passes — worse than no gate at all, because it manufactures
confidence. A hard failure with a clear "you need the key, ask X" message is correct.
Bonus: this makes the repo site-agnostic by construction
The same gate that blocks the site IFO value blocks any hardcoded site channel name. That is
already a stated design requirement — the CDS backend spec says Component 1 must not hard-code a
single site channel name or the site IFO value so that other hardware needs only a new profile
(docs/superpowers/specs/2026-08-03-cds-hardware-backend-design.md, and the site-profile work in
#26). Right now that requirement is enforced by review, and review had let 133 instances through.
It also gives contributors something concrete to reach for. The 2026-08-04 pass established a
placeholder vocabulary for prose — <host>, <remote-path>, <conda-root>, <IFO> for the site
prefix and <MDL> for the subsystem/front-end model element. <MDL> matters beyond redaction: it
stops channel templates in the notes from hardcoding the suspension paradigm, so they read as
"any model" rather than "this one subsystem".
One caveat worth stating so nobody oversells this: for a project whose site is named in the repo,
placeholders reduce greppability, not inferability. Anyone familiar with LIGO CDS can infer the
prefix. The genuine wins are that the values stop being machine-harvestable from a public repo, and
that the code stops depending on them.
Appendix: history rewrite — proposal only, for a maintainer decision
Recorded because the redaction above fixes only the tip. Not executed, and not something to do
unilaterally.
Affected (SHAs and paths are already public; the strings are what must not be restated):
origin/main@f13c293and earlier — site channel prefix, acrossdocs/examples/01–06,
docs/tutorial/closing-the-loop.qmd,src/system_ident/configs/*.yml, five test modules,
experiments/prior_bakeoff/refinement_sweep.py, andnotes/40m-sos-campaign-handoff-2026-07.md
(which additionally carries the site paths and the personal home path).origin/feat/cds-hardware-backend@d9d3432— interpreter version ×3 and site IFO value ×5, in
docs/superpowers/plans/2026-08-03-cds-hardware-backend.md,
docs/superpowers/specs/2026-08-03-cds-hardware-backend-design.md, and
notes/cds-hardware-bringup-2026-08.md.
Shape, if approved:
git filter-repo --replace-text <replacements-file>
The replacements file is generated locally from the gitignored manifest and never committed — the
same paradox as above applies to the rewrite tooling.
Costs, stated honestly:
- Every collaborator must re-clone; stale clones reintroduce the old objects on their next push.
- Open PRs break and need recreating.
- Existing forks keep the old objects, and we cannot reach into them.
- GitHub keeps unreachable commits fetchable by SHA unless Support is asked to purge them, and
the SHAs are in this issue and in the reflogs of anyone who cloned. - Third-party mirrors, the rendered Pages site, and search-engine caches are all outside our control.
So a rewrite lowers future discoverability but does not guarantee retraction, and it imposes real
coordination cost on everyone else.
Recommendation: accept the historical exposure, redact going forward, and land the gate in this
issue so the exposure stops growing. Nothing found here is a credential — no key, token or password
was ever committed. The categories are a service endpoint, an interpreter version, and a naming
prefix. Worth stopping; probably not worth a shared-history rewrite. Reassess if a future finding is
an actual secret.
- Lingua principale
- Python
- Stelle
- 0
- Fork
- 1
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Preparare l'ambiente
Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di CaltechExperimentalGravity/system_ident
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 35/100
-
enhancement
Difficoltà 5/5 Più di una settimana Idoneità per principianti 15/100
-
enhancement
Difficoltà 5/5 Più di una settimana Idoneità per principianti 35/100
-
enhancement
Difficoltà 5/5 Più di una settimana Idoneità per principianti 35/100
-
enhancement
Difficoltà 5/5 Più di una settimana Idoneità per principianti 25/100
Tutte le issue di CaltechExperimentalGravity/system_ident
Issue simili
-
Project submission:Aperta
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 72/100
slavakurilyak/awesome-ai-agents#845 ·
I maintainer di solito rispondono entro 1 giorno
-
tests: pin that the atomic writers in utils/paths.py stage the temporary file in the target's directoryForse già presa Una pull request collegata a questa issue è aperta o già unita. Apertadifficulty:2 enhancement good first issue
Difficoltà 1/5 1-3 ore Idoneità per principianti 92/100
MakazhanAlpamys/Soup#2178 ·
I maintainer di solito rispondono entro 1 giorno
-
[BUG] dcg.waggr permutation p-values are folded twice, so near-zero scores get the smallest p-valuesAperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
scverse/rapids-singlecell#854 ·
I maintainer di solito rispondono entro 1 giorno
-
Slang input: int8_t4_packed and uint8_t4_packed are copied into CrossGLForse già presa @ninglinLiu l’ha presa 1 giorno fa. Apertaarea:frontend backend:slang bug direction:input
Difficoltà 2/5 1-3 ore Idoneità per principianti 70/100
I maintainer di solito rispondono entro 1 giorno
-
random_elements(unique=True) on an OrderedDict: ZeroDivisionError once only zero weights remain, AssertionError with use_weighting=FalseForse già presa Una pull request collegata a questa issue è aperta o già unita. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 85/100
I maintainer di solito rispondono entro 13 giorni