sharkdp/hyperfine

Add an option to save a benchmark run for later comparisons

Open

#607 opened on Jan 28, 2023

View on GitHub
 (6 comments) (11 reactions) (0 assignees)Rust (28,110 stars) (479 forks)batch import
feature-requesthelp wanted

Description

There are times when I want to have some baseline saved that I compare later changes to. It would be convenient if I had a way to save a benchmarking run and then load that to later runs. Something like

$ hyperfine --save-benchmark baseline 'sleep 1'
$ hyperfine --load-benchmark baseline 'sleep 0.5'
{{ display things as though `hyperfine 'sleep 1' 'sleep 0.5'` had been run }}

If you're interested in supporting this then I can work on adding it as a feature

Contributor guide

Tech stack
rust
Domain
cliperformance
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
3-5 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Rust programmingCLI argument parsingJSON serializationUnderstanding of hyperfine's benchmark execution
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
40
Research direction
The issue requests adding save benchmark and load benchmark flags. To implement, modify CLI argument parsing in the main entry point (likely src/main.rs or a cli module). Create a serializable format (e.g., JSON) for benchmark results, referencing existing code that handles benchmark runs. Implement file I/O for saving/loading, and adjust the display logic to compare loaded and new runs. Start by discussing the design in the issue to align with maintainers.