Hacktoberfest 2026: as issues que os mantenedores marcaram para outubro, abertas e boas para iniciantes. Ver issues do Hacktoberfest

BL-23: clip_scope: global default paired with unreviewed clipping_norm: 1.0 may over-clip cache_model_0 reference weights

Fechada
  • Algo errado?
#247 1 comentário 0 reações 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 1 dia

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
4/5
Tempo estimado
3-5 dias
Facilidade para iniciantes
48/100
Tipo de issue
Bug
Clareza
Razoavelmente clara
Status de atividade
Ativa
Stack de tecnologia
machine-learning, python
Domínio
backend, machine-learning

Direção de pesquisa

Read cid_services/client.py, cache_model_0/manifest.json, and tests/test_cache_client_dp.py, then inspect how apply_dp_mechanism and clip_state_dict handle the global clipping bound. Run the mechanism on cache_model_0's trained state_dict and measure its combined L2 norm. If the bound clips weights aggressively, retune the manifest value and add a test for post-clip norm retention; done when the real weights are verified and the test passes.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

Approved bug

Summary

PR #109 (fix/dp-sensitivity) correctly defaults clip_scope to global so that noise_scale = noise_multiplier * clipping_norm is calibrated against the actual L2 bound on the combined gradient norm — the sensitivity-calibration fix itself is sound. But cache_model_0/manifest.json's clipping_norm: 1.0 was never revisited after this change, and no test in PR #109 exercises the new default against cache_model_0's real trained state_dict.

Why this matters

Under the old per_layer scope, clipping_norm: 1.0 bounded each tensor independently to norm 1.0.

Under global, clipping_norm: 1.0 bounds the combined L2 norm of all tensors to 1.0:

‖[tensor_0, tensor_1, …, tensor_n]‖₂ = sqrt(Σ ‖tensor_i‖²) ≤ 1.0

For a model with multiple floating tensors this is a much tighter constraint — sqrt(Σ ‖tensor_i‖²) can easily exceed 1.0 even if no individual tensor's norm changes. The result is that clip_state_dict clips more aggressively than before, which degrades model quality proportionally to how much the combined norm exceeds the bound.

PR #109's new tests (tests/test_cache_client_dp.py) use build_small_state_dict (synthetic tensors), not cache_model_0's real trained weights, so this is unverified.

Fix

  1. Run apply_dp_mechanism / clip_state_dict against cache_model_0's actual trained state_dict under the new global default.
  2. Measure the combined L2 norm of the real weights before clipping.
  3. If 1.0 clips aggressively (combined norm >> 1.0), retune clipping_norm in cache_model_0/manifest.json to a value that allows the real weights to pass through unclipped or only lightly clipped.
  4. Add a test in tests/test_cache_client_dp.py that loads the real state_dict and asserts the combined norm post-clip is within a tolerance of the pre-clip norm.

References

  • Developer/BACK_LOG.md BL-23
  • PR #109 (fix/dp-sensitivity) — sensitivity fix and clip_scope: global default
  • cache_model_0/manifest.json — clipping_norm: 1.0, clip_scope: global
  • tests/test_cache_client_dp.py — build_small_state_dict (synthetic, not real weights)
  • cid_services/client.py — clip_state_dict / apply_dp_mechanism
Linguagem predominante
Python
Estrelas
4
Forks
7
Merge médio
3d 5h
PRs com merge (30d)
34

Preparar o ambiente

Este projeto não oferece contêiner de desenvolvimento, Dockerfile nem guia de contribuição, então a configuração fica por sua conta: comece pelo README e veja nosso guia da primeira contribuição para os passos gerais.

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de InfiniteZeroFoundation/DevNet

Todas as issues de InfiniteZeroFoundation/DevNet

Issues semelhantes

Mais issues de Python

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.