Output the probability of being faster (slower) when compare results
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 35/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- python
- Domain
- performance
Research direction
Start by locating the code that compares two benchmark results and the tests covering comparison output. Implement the probability calculation described in the issue, including the faster and slower cases, and verify that comparison output reports the resulting probability.
Written by the indexing model from the issue text.
Description
When compare two results it would be helpful to output a probability of one result be faster then other.
If times1 and times2 are sets of measured times, then the probability of the first benchmark being faster than the second one is estimated as:
sum(x < y for x in times1 for y in times2)/len(times1)/len(times2)
Actually you can sort one of sets and use binary search for optimization.
- Dominant language
- Python
- Stars
- 967
- Forks
- 112
- Avg merge
- 5d 8h
- Merged PRs (30d)
- 9
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from psf/pyperf
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
Maintainers usually reply within 1 day
-
Improve Type System for `pyperf`Possibly taken @DebanKsahu claimed this 69 days ago. Open
Difficulty 4/5 3-5 days Newbie friendliness 45/100
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 32/100
Maintainers usually reply within 1 day
-
Extend hook's teardown metadata with warmup/calibration flagsPossibly taken @maurycy claimed this 170 days ago. Open
Difficulty 4/5 3-5 days Newbie friendliness 48/100
psf/pyperf#239 · 12 comments ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
Maintainers usually reply within 1 day
Similar issues
-
plot_graph crashes when a mesh node has no m2m edgePossibly taken @tygvi claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
mllam/neural-lam#883 · 1 comment ·
Maintainers usually reply within 2 days
-
enhancement possible-duplicate
Difficulty 1/5 1-3 hours Newbie friendliness 78/100
AOSSIE-Org/PictoPy#1715 · 2 comments ·
Maintainers usually reply within 6 days
-
SQLite connector omits generated columns from the catalog, and drops relationships sourced on themOpen
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
NVIDIA/auto-ontology#511 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
ethereum/execution-specs#3996 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
cpp-best-practices/project_validator#267 ·
Maintainers usually reply within 1 day