sharkdp/hyperfine

disabling clock frequency scaling and dropping page cache

Open

#239 opened on Oct 28, 2019

View on GitHub
 (6 comments) (0 reactions) (0 assignees)Rust (28,110 stars) (479 forks)batch import
help wantedquestion

Description

Hi! Cool utility, @nathanchance is using it to benchmark some of our toolchains.

I was wondering if hyperfine does anything related to disabling clock frequency scaling?

On Android, if we don't pin the cores' clocks to fixed frequency, the noise in the benchmark frequently makes the results unreliable.

set -e;

get_cpus () {
  echo -n "$(adb shell 'ls /sys/devices/system/cpu' | grep 'cpu[0-9]')"
}

set_userspace_freq() {
  #echo "setting us $1"
  shell_command="echo userspace > /sys/devices/system/cpu/$1/cpufreq/scaling_governor"
  echo -n $(adb shell $shell_command)
}

get_max_freq() {
  #echo "looking for $1"
  shell_command="cat /sys/devices/system/cpu/$1/cpufreq/cpuinfo_max_freq"
  echo -n "$(adb shell $shell_command)"
}

set_freq() {
  shell_command="echo $2 > /sys/devices/system/cpu/$1/cpufreq/scaling_setspeed"
  echo -n "$(adb shell $shell_command)"
}

verify() {
   shell_command="cat /sys/devices/system/cpu/$1/cpufreq/cpuinfo_cur_freq"
   actual_freq=$(adb shell $shell_command)
   if [ "$actual_freq" == "$2" ]; then
     echo "good"
   else
     echo "bad"
   fi
}

echo "finding number of cpus"
for cpu in $(get_cpus)
do
    #echo "$cpu"
    echo "setting cpu frequency scaling governor to userspace control"
    set_userspace_freq $cpu
    echo "finding max freqency of $cpu"
    freq="$(get_max_freq $cpu)"
    echo "setting $cpu to $freq"
    set_freq $cpu $freq
    verify $cpu $freq
done

Is what I usually do; I'm not sure if the same sysfs nodes are exposed on x86.

Another thing I do when benchmarking is clear the page cache rather than do warmup runs:

echo 3 > /proc/sys/vm/drop_caches

I couldn't quickly tell if these are done by hyperfine, but maybe they might help?

Contributor guide