Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

DP: default noise_multiplier (σ = 5.0 at clipping_norm 10) reduces the reference model to chance accuracy once DP is enabled

Aperta
  • Qualcosa non va?
#299 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
25/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python
Ambito
machine-learning, security

Direzione di ricerca

Start with tests/test_cache_client_dp.py and the test_manifest_dp_full_noise_impact test, then read cache_model_0/manifest.json and Documentation/public/manifest.md for the shipped DP settings. The issue presents a documentation-only interim option and an accountant-dependent option; the latter needs the DP specialist's input on issue No. 279. Done means documenting the measured utility impact without making an ε claim, or agreeing on an accountant-backed setting and updating the test.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Summary

With the reference manifest's DP block switched on, the default noise wipes out the model. cache_model_0/manifest.json ships noise_multiplier: 0.5 with clipping_norm: 10.0 (raised from 1.0 by PR No. 263 / issue No. 247). The noise scale is noise_multiplier × clipping_norm, so σ = 5.0 per coordinate, and accuracy falls to chance:

Source Baseline Clip only Clip + default noise
PR No. 263 tests (seed 42, 2,000 samples from dataset/test/test_dataset.pt, reproducible) 0.860 0.860 0.120
PR No. 293 write-up (separate experiment, script not committed) 85.25% 85–86% ~13%, at every clipping_norm from 1 to 20, and for update_gaussian too

DP is "enabled": false by default, so nothing breaks today. But a model owner who turns DP on with the shipped parameters gets a useless model and no warning.

Why

Noise is added independently to every floating coordinate. For a model with D parameters, the total noise norm grows as σ·√D. cache_model_0's MLP has ~109K parameters (√D ≈ 330), so the noise swamps the signal regardless of how well clipping_norm is tuned. Raising clipping_norm to stop over-clipping (PR No. 263) also raised σ by the same factor. PR No. 293's sweep (not reproduced here) only recovers utility at noise_multiplier ≈ 0.001.

The current test pins this behaviour rather than flagging it: test_manifest_dp_full_noise_impact (tests/test_cache_client_dp.py) asserts dp_acc < baseline_acc - 0.3. Its docstring already suggests ~0.05 "would restore σ≈0.5" but leaves the choice open.

What needs deciding

Picking noise_multiplier is a privacy-budget decision, not a tuning knob. Documentation/technical/audits/dp-mechanism-review.md puts accountant integration (item 3) and DP-SGD (item 4) ahead of any ε claim. So the options are:

  1. Interim (docs/manifest only): keep DP off by default, but warn clearly in Documentation/public/manifest.md and the manifest example that the shipped noise_multiplier gives no utility for cache_model_0, and give a measured utility-vs-noise table. No ε claim.
  2. With the accountant (issue No. 279 / review item 3): choose a default noise_multiplier together with a reported (ε, δ) and a measured accuracy, and update test_manifest_dp_full_noise_impact to assert acceptable utility at that setting.

Option 1 can land now. Option 2 needs the DP specialist's input on issue No. 279.

References

  • Issue No. 247 (closed), PR No. 263: clipping_norm retune and the real-weight accuracy tests
  • PR No. 293: noise_multiplier sweep (Finding 2), changes requested
  • Issue No. 279: DP mechanism fixes, DP specialist tagged
  • cache_model_0/manifest.json dp.parameters; Documentation/public/manifest.md "Choosing clipping_norm"
Lingua principale
Python
Stelle
4
Fork
7
Merge medio
3g 5h
PR unite (30g)
34

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di InfiniteZeroFoundation/DevNet

Tutte le issue di InfiniteZeroFoundation/DevNet

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.