Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

BL-23: clip_scope: global default paired with unreviewed clipping_norm: 1.0 may over-clip cache_model_0 reference weights

Chiusa
  • Qualcosa non va?
#247 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
48/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
machine-learning, python
Ambito
backend, machine-learning

Direzione di ricerca

Read cid_services/client.py, cache_model_0/manifest.json, and tests/test_cache_client_dp.py, then inspect how apply_dp_mechanism and clip_state_dict handle the global clipping bound. Run the mechanism on cache_model_0's trained state_dict and measure its combined L2 norm. If the bound clips weights aggressively, retune the manifest value and add a test for post-clip norm retention; done when the real weights are verified and the test passes.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Approved bug

Summary

PR #109 (fix/dp-sensitivity) correctly defaults clip_scope to global so that noise_scale = noise_multiplier * clipping_norm is calibrated against the actual L2 bound on the combined gradient norm — the sensitivity-calibration fix itself is sound. But cache_model_0/manifest.json's clipping_norm: 1.0 was never revisited after this change, and no test in PR #109 exercises the new default against cache_model_0's real trained state_dict.

Why this matters

Under the old per_layer scope, clipping_norm: 1.0 bounded each tensor independently to norm 1.0.

Under global, clipping_norm: 1.0 bounds the combined L2 norm of all tensors to 1.0:

‖[tensor_0, tensor_1, …, tensor_n]‖₂ = sqrt(Σ ‖tensor_i‖²) ≤ 1.0

For a model with multiple floating tensors this is a much tighter constraint — sqrt(Σ ‖tensor_i‖²) can easily exceed 1.0 even if no individual tensor's norm changes. The result is that clip_state_dict clips more aggressively than before, which degrades model quality proportionally to how much the combined norm exceeds the bound.

PR #109's new tests (tests/test_cache_client_dp.py) use build_small_state_dict (synthetic tensors), not cache_model_0's real trained weights, so this is unverified.

Fix

  1. Run apply_dp_mechanism / clip_state_dict against cache_model_0's actual trained state_dict under the new global default.
  2. Measure the combined L2 norm of the real weights before clipping.
  3. If 1.0 clips aggressively (combined norm >> 1.0), retune clipping_norm in cache_model_0/manifest.json to a value that allows the real weights to pass through unclipped or only lightly clipped.
  4. Add a test in tests/test_cache_client_dp.py that loads the real state_dict and asserts the combined norm post-clip is within a tolerance of the pre-clip norm.

References

  • Developer/BACK_LOG.md BL-23
  • PR #109 (fix/dp-sensitivity) — sensitivity fix and clip_scope: global default
  • cache_model_0/manifest.json — clipping_norm: 1.0, clip_scope: global
  • tests/test_cache_client_dp.py — build_small_state_dict (synthetic, not real weights)
  • cid_services/client.py — clip_state_dict / apply_dp_mechanism
Lingua principale
Python
Stelle
4
Fork
7
Merge medio
3g 4h
PR unite (30g)
34

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di InfiniteZeroFoundation/DevNet

Tutte le issue di InfiniteZeroFoundation/DevNet

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.