Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

[Phase 16] S10 — Data-layer fidelity, scale and privacy

Open
  • Is something wrong?
#896 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
10/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
python
Domain
data, security

Research direction

This is a multi-unit roadmap tracker, not a single fix. Start with docs/roadmap/phase-16-trust-surface-hardening.md (lines 583-611) and CLAUDE.md for the gauntlet. The public functions find_near_duplicates and find_near_duplicates_minhash in data_audit.all must keep their pair-list shape, so changes must be additive. Done means the listed exit criteria pass, each with its own red-then-green reproducer.

Written by the indexing model from the issue text.

Description

phase: 16 source: roadmap

Roadmap · Phase 16, step S10 · size L · 7 unit(s) · planned in docs/roadmap/phase-16-trust-surface-hardening.md:583-611

Summary

Two threads that share one report schema. The data layer reports a clean or complete result it did not compute — dropped Presidio rows, a no-op script-sanity check, silent decode replacement, a duplicate scan that cannot finish — and a declared data intent is silently not honoured (multimodal.text_column is dead; mix_ratio does not produce the requested blend). CORE-13 is the same layer's privacy contract: a synthetic empty-response WARNING logs raw prompt content.

Units

Unit Tracked by (October 2026 review)
F-W20260729-TRUST-06 #348 · #547 · #601 · #610 · #652
F-W20260729-TRUST-09 #348 · #547 · #551 · #601 · #610
F-W20260729-TRUST-11 #348 · #547 · #601 · #610
F-W20260729-TRUST-12 #348 · #478 · #547 · #601 · #610
F-W20260729-CORE-08 #399 · #588 · #596 · #621 · #739
F-W20260729-CORE-11 #545 · #559 · #588 · #596 · #621 · #688
F-W20260729-CORE-13 #699

Plan

  • find_near_duplicates and find_near_duplicates_minhash are in data_audit.__all__, so the pair-list shape is public API and the fix must be additive.
  • Group on the exact fingerprint before LSH, so k identical rows contribute k(k-1)/2 arithmetically instead of materialising k(k-1)/2 tuples.

Exit criteria

  • 5,000 identical records still yield exactly 12,497,500 pairs with subquadratic peak memory under tracemalloc, and a property test holds count_… == len(find_…) across random inputs, including the brute-force fallback.
  • An every-row Presidio failure yields coverage=0, incomplete=true.
  • tr / TR / Tr normalise byte-identically.
  • A single invalid byte in a long file reports decode_replacements=1.
  • A text_column absent from the dataset fails early.
  • Realised mix ratios and row counts appear in the result artefact.
  • A caplog test proves known PII tokens appear in no log record, while the warning still identifies the prompt by index and length.

How to deliver

  • One root-cause family, committed on its own; then an Opus review round and a Sonnet review round, each with verified findings fixed and committed before the next step begins.
  • A reproducer that is red before the fix and green after; the full gauntlet from CLAUDE.md passes under ./.venv/bin/python.
  • Every English documentation edit ships with its Turkish mirror in the same commit; a new or renamed audit event gets its EN and TR catalog rows in the same PR.
  • A step that grows a deferred module carries its budget_history justification in the same diff; a widened guard rolls out non-strict first, then flips.
  • Commit bodies cite Absorbs F-W20260729-…; the CHANGELOG [Unreleased] section gets an entry only for a change a user or library consumer can observe (decision C-20).

Planned in docs/roadmap/phase-16-trust-surface-hardening.md (read at f94595f). Unit IDs (F-W20260729-…) come from the 2026-07-29/30 full-project review; the issue numbers next to them are the October 2026 review's records of the same defects.

Dominant language
Python
Stars
9
Forks
1
Avg merge
8h 10m
Merged PRs (30d)
5

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from HodeTech/ForgeLM

All issues in HodeTech/ForgeLM

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.