TUI wordlist load and resume digests read whole files into memory

未关闭 适合新手
#411 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
2/5
预计耗时
1-3 小时
新手友好度
82/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
活跃
技术栈
go
领域
cli, performance

调研方向

阅读 internal/tui/model.go:212-221 和 internal/cli/resume.go:368-378,然后对比 internal/cli/wordlist.go:109 中现有的流式处理路径和内置列表分支。更新 TUI 文件加载和 fileDigest,使其以流式方式处理输入,同时保留 digest 输出。运行 go test ./internal/tui/ ./internal/cli/;当两个验收标准都通过且现有的恢复状态文件仍可正常使用时,即表示完成。

由索引模型根据 Issue 内容生成。

描述

good first issue performance severity: low

Summary

Two paths still read a whole input file into memory, which the streaming wordlist reader (#248) otherwise avoids: the TUI's wordlist load, and the digests a resume state records.

Where

  • internal/tui/model.go:212-216 (wordlist.LoadWordlist, which is ReadLines + Build)
  • internal/wordlist/reader.go:391-398
  • internal/cli/resume.go:368-378 (fileDigest: os.ReadFile + sha256.Sum256)

Details

  • TUI. For a wordlist file the TUI calls LoadWordlist, which appends every raw line to a []string before normalising. The CLI streams the same file with wordlist.EachLineOf(p, pr.Add) (internal/cli/wordlist.go:109). So does the TUI's own bundled-list branch three lines below (model.go:217-221).
  • Resume digests. fileDigest reads every -w, -dL, -exclude-file, -r, -seeds and -diff file whole, when a stopped run saves its state and again when -resume checks its inputs.

Impact

  • A 5M-line list costs the TUI about 80-160 MB of extra peak memory.
  • A 50-150 MB wordlist is loaded whole at Ctrl+C time and again at resume.

Suggested fix

  • TUI: pr := wordlist.NewPreparer(); wordlist.EachLineOf(vals.wordlist, pr.Add); pr.Prepared().For(vals.Domain), as the bundled branch already does.
  • fileDigest: f, _ := os.Open(path); h := sha256.New(); io.Copy(h, f).

Acceptance criteria

  • The TUI loads wordlist files through EachLineOf
  • fileDigest streams, and its digests are unchanged, so existing state files still resume

Getting started

  • Relevant files: internal/tui/model.go, internal/cli/resume.go
  • How to run tests: go test ./internal/tui/ ./internal/cli/
  • Related: #248

Filed by repo-audit/full-audit at dd5b156. Severity: low. Effort: S. audit-key: performance|internal/tui/model.go|loadWordlist

主要语言
Go
星标
1
派生
1
平均合并
7 小时 18 分钟
30 天内合并 PR
23

环境准备

在 Codespaces 中打开

在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

TMHSDigital/subenum 的其他 Issue

查看 TMHSDigital/subenum 的全部 Issue

相似的 Issue

更多 Go Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。