Reconsidering min()?
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- python
- Domain
- performance
Research direction
Start by reproducing the reported timeit.repeat distribution and compare it with the mean and standard deviation that pyperf reports. Read the benchmark-statistics documentation and implementation to determine whether slow runs are excluded; done means the behavior is verified and the issue has a clear decision or documented explanation.
Written by the indexing model from the issue text.
Description
I've done a little experiment: I run timeit.repeat 1000 times on a really simple statement. After that, I sorted the list and I grouped the results in time slices, counting the number of times a bench is inside a certain time slice. This way I have a sort of probability. Then I plot the result:
https://www.dropbox.com/s/f4naryc055k42cs/2020-07-24-080340_1366x768_scrot.png?dl=0
The result is quite interesting. As you can see, the distribution is not a normal distribution. It seems a multimodal normal distribution. Furthermore, the highest probability is near the minimum value, and not near the average (that is on the second peak).
Mean and stdev are only meaningful if the distribution is a normal distribution. It seems that the way you can have a sort of normal distribution is to consider only the first part of the plot.
Since pyperf returns mean and stdev, does it "cut" the slowest runs?
- Dominant language
- Python
- Stars
- 967
- Forks
- 112
- Avg merge
- 5d 8h
- Merged PRs (30d)
- 9
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from psf/pyperf
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
Maintainers usually reply within 1 day
-
Improve Type System for `pyperf`Possibly taken @DebanKsahu claimed this 69 days ago. Open
Difficulty 4/5 3-5 days Newbie friendliness 45/100
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 32/100
Maintainers usually reply within 1 day
-
Extend hook's teardown metadata with warmup/calibration flagsPossibly taken @maurycy claimed this 170 days ago. Open
Difficulty 4/5 3-5 days Newbie friendliness 48/100
psf/pyperf#239 · 12 comments ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
Maintainers usually reply within 1 day
Similar issues
-
plot_graph crashes when a mesh node has no m2m edgePossibly taken @tygvi claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
mllam/neural-lam#883 · 1 comment ·
Maintainers usually reply within 2 days
-
enhancement possible-duplicate
Difficulty 1/5 1-3 hours Newbie friendliness 78/100
AOSSIE-Org/PictoPy#1715 · 2 comments ·
Maintainers usually reply within 6 days
-
SQLite connector omits generated columns from the catalog, and drops relationships sourced on themOpen
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
NVIDIA/auto-ontology#511 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
ethereum/execution-specs#3996 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
cpp-best-practices/project_validator#267 ·
Maintainers usually reply within 1 day