Hacktoberfest 2026：维护者为十月标记出来的 issue，仍然开放、适合新手。 浏览 Hacktoberfest issue

[Phase 16] S10 — Data-layer fidelity, scale and privacy

未关闭
  • 哪里不对？
#896 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

评估

难度
5/5
预计耗时
一周以上
新手友好度
10/100
Issue 类型
缺陷
描述清晰度
基本清楚
活跃度
活跃
技术栈
python
领域
data, security

调研方向

This is a multi-unit roadmap tracker, not a single fix. Start with docs/roadmap/phase-16-trust-surface-hardening.md (lines 583-611) and CLAUDE.md for the gauntlet. The public functions find_near_duplicates and find_near_duplicates_minhash in data_audit.all must keep their pair-list shape, so changes must be additive. Done means the listed exit criteria pass, each with its own red-then-green reproducer.

由索引模型根据 Issue 内容生成。

描述

phase: 16 source: roadmap

Roadmap · Phase 16, step S10 · size L · 7 unit(s) · planned in docs/roadmap/phase-16-trust-surface-hardening.md:583-611

Summary

Two threads that share one report schema. The data layer reports a clean or complete result it did not compute — dropped Presidio rows, a no-op script-sanity check, silent decode replacement, a duplicate scan that cannot finish — and a declared data intent is silently not honoured (multimodal.text_column is dead; mix_ratio does not produce the requested blend). CORE-13 is the same layer's privacy contract: a synthetic empty-response WARNING logs raw prompt content.

Units

Unit Tracked by (October 2026 review)
F-W20260729-TRUST-06 #348 · #547 · #601 · #610 · #652
F-W20260729-TRUST-09 #348 · #547 · #551 · #601 · #610
F-W20260729-TRUST-11 #348 · #547 · #601 · #610
F-W20260729-TRUST-12 #348 · #478 · #547 · #601 · #610
F-W20260729-CORE-08 #399 · #588 · #596 · #621 · #739
F-W20260729-CORE-11 #545 · #559 · #588 · #596 · #621 · #688
F-W20260729-CORE-13 #699

Plan

  • find_near_duplicates and find_near_duplicates_minhash are in data_audit.__all__, so the pair-list shape is public API and the fix must be additive.
  • Group on the exact fingerprint before LSH, so k identical rows contribute k(k-1)/2 arithmetically instead of materialising k(k-1)/2 tuples.

Exit criteria

  • 5,000 identical records still yield exactly 12,497,500 pairs with subquadratic peak memory under tracemalloc, and a property test holds count_… == len(find_…) across random inputs, including the brute-force fallback.
  • An every-row Presidio failure yields coverage=0, incomplete=true.
  • tr / TR / Tr normalise byte-identically.
  • A single invalid byte in a long file reports decode_replacements=1.
  • A text_column absent from the dataset fails early.
  • Realised mix ratios and row counts appear in the result artefact.
  • A caplog test proves known PII tokens appear in no log record, while the warning still identifies the prompt by index and length.

How to deliver

  • One root-cause family, committed on its own; then an Opus review round and a Sonnet review round, each with verified findings fixed and committed before the next step begins.
  • A reproducer that is red before the fix and green after; the full gauntlet from CLAUDE.md passes under ./.venv/bin/python.
  • Every English documentation edit ships with its Turkish mirror in the same commit; a new or renamed audit event gets its EN and TR catalog rows in the same PR.
  • A step that grows a deferred module carries its budget_history justification in the same diff; a widened guard rolls out non-strict first, then flips.
  • Commit bodies cite Absorbs F-W20260729-…; the CHANGELOG [Unreleased] section gets an entry only for a change a user or library consumer can observe (decision C-20).

Planned in docs/roadmap/phase-16-trust-surface-hardening.md (read at f94595f). Unit IDs (F-W20260729-…) come from the 2026-07-29/30 full-project review; the issue numbers next to them are the October 2026 review's records of the same defects.

主要语言
Python
星标
9
派生
1
平均合并
4 小时 3 分钟
30 天内合并 PR
3

环境准备

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

HodeTech/ForgeLM 的其他 Issue

查看 HodeTech/ForgeLM 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。