Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

BL-23: clip_scope: global default paired with unreviewed clipping_norm: 1.0 may over-clip cache_model_0 reference weights

Cerrado
  • ¿Algo no está bien?
#247 1 comentario 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
48/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
machine-learning, python
Área
backend, machine-learning

Línea de trabajo

Read cid_services/client.py, cache_model_0/manifest.json, and tests/test_cache_client_dp.py, then inspect how apply_dp_mechanism and clip_state_dict handle the global clipping bound. Run the mechanism on cache_model_0's trained state_dict and measure its combined L2 norm. If the bound clips weights aggressively, retune the manifest value and add a test for post-clip norm retention; done when the real weights are verified and the test passes.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Approved bug

Summary

PR #109 (fix/dp-sensitivity) correctly defaults clip_scope to global so that noise_scale = noise_multiplier * clipping_norm is calibrated against the actual L2 bound on the combined gradient norm — the sensitivity-calibration fix itself is sound. But cache_model_0/manifest.json's clipping_norm: 1.0 was never revisited after this change, and no test in PR #109 exercises the new default against cache_model_0's real trained state_dict.

Why this matters

Under the old per_layer scope, clipping_norm: 1.0 bounded each tensor independently to norm 1.0.

Under global, clipping_norm: 1.0 bounds the combined L2 norm of all tensors to 1.0:

‖[tensor_0, tensor_1, …, tensor_n]‖₂ = sqrt(Σ ‖tensor_i‖²) ≤ 1.0

For a model with multiple floating tensors this is a much tighter constraint — sqrt(Σ ‖tensor_i‖²) can easily exceed 1.0 even if no individual tensor's norm changes. The result is that clip_state_dict clips more aggressively than before, which degrades model quality proportionally to how much the combined norm exceeds the bound.

PR #109's new tests (tests/test_cache_client_dp.py) use build_small_state_dict (synthetic tensors), not cache_model_0's real trained weights, so this is unverified.

Fix

  1. Run apply_dp_mechanism / clip_state_dict against cache_model_0's actual trained state_dict under the new global default.
  2. Measure the combined L2 norm of the real weights before clipping.
  3. If 1.0 clips aggressively (combined norm >> 1.0), retune clipping_norm in cache_model_0/manifest.json to a value that allows the real weights to pass through unclipped or only lightly clipped.
  4. Add a test in tests/test_cache_client_dp.py that loads the real state_dict and asserts the combined norm post-clip is within a tolerance of the pre-clip norm.

References

  • Developer/BACK_LOG.md BL-23
  • PR #109 (fix/dp-sensitivity) — sensitivity fix and clip_scope: global default
  • cache_model_0/manifest.json — clipping_norm: 1.0, clip_scope: global
  • tests/test_cache_client_dp.py — build_small_state_dict (synthetic, not real weights)
  • cid_services/client.py — clip_state_dict / apply_dp_mechanism
Lenguaje dominante
Python
Estrellas
4
Forks
7
Merge medio
3 d 4 h
PR fusionados (30 d)
34

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de InfiniteZeroFoundation/DevNet

Todos los issues de InfiniteZeroFoundation/DevNet

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.