perf(scan): AVX-512 + vpclmulqdq scanner backend
还没有人认领这个 Issue。
评估
- 难度
- 5/5
- 预计耗时
- 一周以上
- 新手友好度
- 30/100
- Issue 类型
- 功能
- 描述清晰度
- 基本清楚
- 活跃度
- 冷清
- 技术栈
- github-actions, rust
- 领域
- ci-cd, performance
调研方向
Read src/scan/avx2.rs and src/scan/mod.rs, then run the proposed CPU-feature diagnostic in the CI workflow to establish whether vpclmulqdq is consistently available. If the prerequisite is met, done means adding the feature-gated AVX-512 path, dispatcher fallback, cross-check coverage, supported-runner validation, and benchmark evidence.
由索引模型根据 Issue 内容生成。
描述
Context
Current scanner uses AVX2 + PCLMUL (128-bit). On CPUs supporting avx512bw + vpclmulqdq (Ice Lake / Sapphire Rapids / Zen 4+), a 128-byte chunk path could halve the loop iteration count.
Prerequisite: CPU support audit
This issue is gated on confirming that the project's actual build/CI hosts support vpclmulqdq. If not, ROI is 0 and the issue should be deferred indefinitely.
-
Local dev host: confirmed missing
vpclmulqdq(Skylake-X / Skylake-SP — has avx512bw but not vpclmulqdq). Cannot test locally. -
CI runners:
ubuntu-latestrunner CPUs vary by allocation. Add a one-line diagnostic to the workflow:- name: CPU features run: grep -oE '\b(avx2|avx512bw|vpclmulqdq|pclmulqdq)\b' /proc/cpuinfo | sort -uCollect output over several CI runs; only proceed if
vpclmulqdqis reliably present.
If CI runners do not reliably provide vpclmulqdq, the only path to validating this is paid larger-runners or self-hosted runners.
Proposal (pending CPU confirmation)
- New
src/scan/avx512.rsmirroringavx2.rswith 128-byte chunks - Dispatcher (
src/scan/mod.rs): AVX-512 → AVX2 → scalar fallback chain - New
avx512feature flag (default off) so release builds stay portable - Use
_mm512_clmulepi64_epi128for the inside-string prefix-XOR
Estimated impact
| est. speedup | |
|---|---|
CPUs with avx512bw + vpclmulqdq |
~1.5–2× scan throughput |
| Other CPUs | 0 (dispatcher falls back) |
Validation plan
-
scanner_crosscheckproptest extended to compare AVX-512 vs scalar - CI matrix on a runner confirmed to have
vpclmulqdq -
make bench3-run median on supported hardware
Recommendation
Last in the perf followup queue. The CPU support situation is uncertain; if it turns out CI runners don't have vpclmulqdq, this is dead code we maintain forever. Do the cheap wins (#5 memchr, #6 pooling, #7 PGO, #8 micro-opts) first.
- 主要语言
- Rust
- 星标
- 2
- 派生
- 0
- PR 合并指标
- 30 天内没有已合并 PR
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
api7/lua-qjson 的其他 Issue
-
docs: fix malformed benchmark throughput table可能重新可做 关联的 PR 已关闭且未合并。 未关闭
难度 1/5 1 小时以内 新手友好度 90/100
-
enhancement
难度 5/5 一周以上 新手友好度 35/100
相似的 Issue
-
enhancement
难度 2/5 1-3 小时 新手友好度 78/100
zcashlabs/thus-spoke-zakura#153 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 79/100
topgrade-rs/topgrade#2395 ·
维护者通常 1 天内回复
-
app bug windows-os
难度 2/5 1-3 小时 新手友好度 67/100
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 88/100
matrix-org/matrix-rust-sdk#7217 ·
维护者通常 1 天内回复
-
editor good first issue
难度 2/5 1-3 小时 新手友好度 65/100
funnyboy-roks/inq#54 ·