Hacktoberfest 2026: as issues que os mantenedores marcaram para outubro, abertas e boas para iniciantes. Ver issues do Hacktoberfest

DP: default noise_multiplier (σ = 5.0 at clipping_norm 10) reduces the reference model to chance accuracy once DP is enabled

Aberta
  • Algo errado?
#299 1 comentário 0 reações 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 1 dia

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
5/5
Tempo estimado
Mais de uma semana
Facilidade para iniciantes
25/100
Tipo de issue
Bug
Clareza
Razoavelmente clara
Status de atividade
Ativa
Stack de tecnologia
python
Domínio
machine-learning, security

Direção de pesquisa

Start with tests/test_cache_client_dp.py and the test_manifest_dp_full_noise_impact test, then read cache_model_0/manifest.json and Documentation/public/manifest.md for the shipped DP settings. The issue presents a documentation-only interim option and an accountant-dependent option; the latter needs the DP specialist's input on issue No. 279. Done means documenting the measured utility impact without making an ε claim, or agreeing on an accountant-backed setting and updating the test.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

Summary

With the reference manifest's DP block switched on, the default noise wipes out the model. cache_model_0/manifest.json ships noise_multiplier: 0.5 with clipping_norm: 10.0 (raised from 1.0 by PR No. 263 / issue No. 247). The noise scale is noise_multiplier × clipping_norm, so σ = 5.0 per coordinate, and accuracy falls to chance:

Source Baseline Clip only Clip + default noise
PR No. 263 tests (seed 42, 2,000 samples from dataset/test/test_dataset.pt, reproducible) 0.860 0.860 0.120
PR No. 293 write-up (separate experiment, script not committed) 85.25% 85–86% ~13%, at every clipping_norm from 1 to 20, and for update_gaussian too

DP is "enabled": false by default, so nothing breaks today. But a model owner who turns DP on with the shipped parameters gets a useless model and no warning.

Why

Noise is added independently to every floating coordinate. For a model with D parameters, the total noise norm grows as σ·√D. cache_model_0's MLP has ~109K parameters (√D ≈ 330), so the noise swamps the signal regardless of how well clipping_norm is tuned. Raising clipping_norm to stop over-clipping (PR No. 263) also raised σ by the same factor. PR No. 293's sweep (not reproduced here) only recovers utility at noise_multiplier ≈ 0.001.

The current test pins this behaviour rather than flagging it: test_manifest_dp_full_noise_impact (tests/test_cache_client_dp.py) asserts dp_acc < baseline_acc - 0.3. Its docstring already suggests ~0.05 "would restore σ≈0.5" but leaves the choice open.

What needs deciding

Picking noise_multiplier is a privacy-budget decision, not a tuning knob. Documentation/technical/audits/dp-mechanism-review.md puts accountant integration (item 3) and DP-SGD (item 4) ahead of any ε claim. So the options are:

  1. Interim (docs/manifest only): keep DP off by default, but warn clearly in Documentation/public/manifest.md and the manifest example that the shipped noise_multiplier gives no utility for cache_model_0, and give a measured utility-vs-noise table. No ε claim.
  2. With the accountant (issue No. 279 / review item 3): choose a default noise_multiplier together with a reported (ε, δ) and a measured accuracy, and update test_manifest_dp_full_noise_impact to assert acceptable utility at that setting.

Option 1 can land now. Option 2 needs the DP specialist's input on issue No. 279.

References

  • Issue No. 247 (closed), PR No. 263: clipping_norm retune and the real-weight accuracy tests
  • PR No. 293: noise_multiplier sweep (Finding 2), changes requested
  • Issue No. 279: DP mechanism fixes, DP specialist tagged
  • cache_model_0/manifest.json dp.parameters; Documentation/public/manifest.md "Choosing clipping_norm"
Linguagem predominante
Python
Estrelas
4
Forks
7
Merge médio
3d 4h
PRs com merge (30d)
34

Preparar o ambiente

Este projeto não oferece contêiner de desenvolvimento, Dockerfile nem guia de contribuição, então a configuração fica por sua conta: comece pelo README e veja nosso guia da primeira contribuição para os passos gerais.

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de InfiniteZeroFoundation/DevNet

Todas as issues de InfiniteZeroFoundation/DevNet

Issues semelhantes

Mais issues de Python

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.