Performance
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Domain
- devops, operating-systems, performance
Research direction
Start by reviewing the linked LLVM, CPU frequency scaling, and pyperf system-tuning guides, then inspect the tentative Nushell commands and the listed ASV --cpu-affinity and taskset entry points. Done should mean the benchmark environment can automatically validate or apply the documented per-boot and per-run setup.
Written by the indexing model from the issue text.
Description
For improved reliability, the machine needs some setup. Maybe we can validate that automatically?
Guides:
- https://google.github.io/benchmark/reducing_variance.html
- https://juliaci.github.io/BenchmarkTools.jl/stable/linuxtips/
- https://llvm.org/docs/Benchmarking.html
- https://wiki.archlinux.org/title/CPU_frequency_scaling
-
sudo python -mpyperf system tune
Once (total, in BIOS)
- Disable Hyperthreading/SMT (looks complex, prefer doing in BIOS)
Per boot
-
Global setup:
Use performance governor for all CPUS
Disable processor boosting
Reduce swappiness
No ASLRtentative script so far:
# performance governor echo "performance" | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor # no boosting echo 0 | sudo tee /sys/devices/system/cpu/cpufreq/boost # low swappiness sudo sysctl vm.swappiness=10 # no ASLR sudo sysctl kernel.randomize_va_space=0 -
Assign roles to CPUS:
- Make sure no timed services run on the CPUs used in benchmarks (processor shielding)
- Set up interrupts (SMP affinity)
Per run
- Set the benchmark program’s task affinity to a fixed cpu. (ASV:
--cpu-affinity, subprocess:taskset -c 0 ./mybenchmark)
- Dominant language
- Rust
- Stars
- 4
- Forks
- 0
- Avg merge
- 5h 52m
- Merged PRs (30d)
- 4
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from scverse/benchmark
-
bug
Difficulty 3/5 1-2 days Newbie friendliness 35/100
-
Difficulty 5/5 Over a week Newbie friendliness 15/100
-
PR runs
Difficulty 5/5 Over a week Newbie friendliness 25/100
-
PR runs Timeline
Difficulty 5/5 Over a week Newbie friendliness 25/100
-
Support OpenMeta
Difficulty 5/5 Over a week Newbie friendliness 20/100
All issues in scverse/benchmark
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Eynzof/Hermes-CN-Desktop#610 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
gitbutlerapp/gitbutler#15998 · 1 comment ·
-
bug triage:deciding
Difficulty 1/5 Under an hour Newbie friendliness 88/100
open-telemetry/otel-arrow#4132 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100