Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

perf(scan): AVX-512 + vpclmulqdq scanner backend

Open
#9 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
30/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Quiet
Tech stack
github-actions, rust
Domain
ci-cd, performance

Research direction

Read src/scan/avx2.rs and src/scan/mod.rs, then run the proposed CPU-feature diagnostic in the CI workflow to establish whether vpclmulqdq is consistently available. If the prerequisite is met, done means adding the feature-gated AVX-512 path, dispatcher fallback, cross-check coverage, supported-runner validation, and benchmark evidence.

Written by the indexing model from the issue text.

Description

enhancement

Context

Current scanner uses AVX2 + PCLMUL (128-bit). On CPUs supporting avx512bw + vpclmulqdq (Ice Lake / Sapphire Rapids / Zen 4+), a 128-byte chunk path could halve the loop iteration count.

Prerequisite: CPU support audit

This issue is gated on confirming that the project's actual build/CI hosts support vpclmulqdq. If not, ROI is 0 and the issue should be deferred indefinitely.

  • Local dev host: confirmed missing vpclmulqdq (Skylake-X / Skylake-SP — has avx512bw but not vpclmulqdq). Cannot test locally.

  • CI runners: ubuntu-latest runner CPUs vary by allocation. Add a one-line diagnostic to the workflow:

    - name: CPU features
      run: grep -oE '\b(avx2|avx512bw|vpclmulqdq|pclmulqdq)\b' /proc/cpuinfo | sort -u
    

    Collect output over several CI runs; only proceed if vpclmulqdq is reliably present.

If CI runners do not reliably provide vpclmulqdq, the only path to validating this is paid larger-runners or self-hosted runners.

Proposal (pending CPU confirmation)

  • New src/scan/avx512.rs mirroring avx2.rs with 128-byte chunks
  • Dispatcher (src/scan/mod.rs): AVX-512 → AVX2 → scalar fallback chain
  • New avx512 feature flag (default off) so release builds stay portable
  • Use _mm512_clmulepi64_epi128 for the inside-string prefix-XOR

Estimated impact

est. speedup
CPUs with avx512bw + vpclmulqdq ~1.5–2× scan throughput
Other CPUs 0 (dispatcher falls back)

Validation plan

  • scanner_crosscheck proptest extended to compare AVX-512 vs scalar
  • CI matrix on a runner confirmed to have vpclmulqdq
  • make bench 3-run median on supported hardware

Recommendation

Last in the perf followup queue. The CPU support situation is uncertain; if it turns out CI runners don't have vpclmulqdq, this is dead code we maintain forever. Do the cheap wins (#5 memchr, #6 pooling, #7 PGO, #8 micro-opts) first.

Dominant language
Rust
Stars
2
Forks
0
PR merge metrics
No merged PRs in 30d

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from api7/lua-qjson

All issues in api7/lua-qjson

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.