Hacktoberfest 2026：维护者为十月标记出来的 issue，仍然开放、适合新手。 浏览 Hacktoberfest issue

BL-23: clip_scope: global default paired with unreviewed clipping_norm: 1.0 may over-clip cache_model_0 reference weights

已关闭
  • 哪里不对？
#247 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
4/5
预计耗时
3-5 天
新手友好度
48/100
Issue 类型
缺陷
描述清晰度
基本清楚
活跃度
活跃
技术栈
machine-learning, python
领域
backend, machine-learning

调研方向

Read cid_services/client.py, cache_model_0/manifest.json, and tests/test_cache_client_dp.py, then inspect how apply_dp_mechanism and clip_state_dict handle the global clipping bound. Run the mechanism on cache_model_0's trained state_dict and measure its combined L2 norm. If the bound clips weights aggressively, retune the manifest value and add a test for post-clip norm retention; done when the real weights are verified and the test passes.

由索引模型根据 Issue 内容生成。

描述

Approved bug

Summary

PR #109 (fix/dp-sensitivity) correctly defaults clip_scope to global so that noise_scale = noise_multiplier * clipping_norm is calibrated against the actual L2 bound on the combined gradient norm — the sensitivity-calibration fix itself is sound. But cache_model_0/manifest.json's clipping_norm: 1.0 was never revisited after this change, and no test in PR #109 exercises the new default against cache_model_0's real trained state_dict.

Why this matters

Under the old per_layer scope, clipping_norm: 1.0 bounded each tensor independently to norm 1.0.

Under global, clipping_norm: 1.0 bounds the combined L2 norm of all tensors to 1.0:

‖[tensor_0, tensor_1, …, tensor_n]‖₂ = sqrt(Σ ‖tensor_i‖²) ≤ 1.0

For a model with multiple floating tensors this is a much tighter constraint — sqrt(Σ ‖tensor_i‖²) can easily exceed 1.0 even if no individual tensor's norm changes. The result is that clip_state_dict clips more aggressively than before, which degrades model quality proportionally to how much the combined norm exceeds the bound.

PR #109's new tests (tests/test_cache_client_dp.py) use build_small_state_dict (synthetic tensors), not cache_model_0's real trained weights, so this is unverified.

Fix

  1. Run apply_dp_mechanism / clip_state_dict against cache_model_0's actual trained state_dict under the new global default.
  2. Measure the combined L2 norm of the real weights before clipping.
  3. If 1.0 clips aggressively (combined norm >> 1.0), retune clipping_norm in cache_model_0/manifest.json to a value that allows the real weights to pass through unclipped or only lightly clipped.
  4. Add a test in tests/test_cache_client_dp.py that loads the real state_dict and asserts the combined norm post-clip is within a tolerance of the pre-clip norm.

References

  • Developer/BACK_LOG.md BL-23
  • PR #109 (fix/dp-sensitivity) — sensitivity fix and clip_scope: global default
  • cache_model_0/manifest.json — clipping_norm: 1.0, clip_scope: global
  • tests/test_cache_client_dp.py — build_small_state_dict (synthetic, not real weights)
  • cid_services/client.py — clip_state_dict / apply_dp_mechanism
主要语言
Python
星标
4
派生
7
平均合并
3 天 4 小时
30 天内合并 PR
34

环境准备

这个项目没有提供开发容器、Dockerfile 或贡献指南，环境需要你自己搭建：先看它的 README，通用步骤见我们的新手贡献指南。

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

InfiniteZeroFoundation/DevNet 的其他 Issue

查看 InfiniteZeroFoundation/DevNet 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。