Increase default running time per process?
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 20/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- python
- Domain
- performance
Research direction
No files or tests are named in the issue. Begin by locating where pyperf chooses default worker counts and durations, then review how results are aggregated and process-to-process comparisons are handled. Done means an agreed default configuration is implemented consistently for JIT and non-JIT implementations and validated by relevant benchmark tests.
Written by the indexing model from the issue text.
Description
Using the default settings, pyperf aims to run 20 worker processes for ~600ms each. Or for implementations that are noted as having jits, 6 processes for 1600ms each.
Is there a strong reason for running so many subprocesses for such a short amount of time? It looks like the results are aggregated and process-to-process comparisons are dropped. 600ms/1600ms is a short amount of time when it comes to jit warmup and in my view doesn't quite reflect the typical experience that users have.
I'd like to propose a new set of numbers, such as 3 worker processes for 4s each. (I'd even be in support of 1 worker process for 12s.) I'd also like to propose using this configuration regardless of whether the implementation has a jit, since I put a higher weight on consistency than using more processes when possible.
What do you all think? I'm also curious what the cinder folks think, I saw @Orvid comment about this in https://github.com/facebookincubator/cinder/issues/74#issuecomment-1128033247
- Dominant language
- Python
- Stars
- 967
- Forks
- 112
- Avg merge
- 5d 8h
- Merged PRs (30d)
- 9
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from psf/pyperf
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
Maintainers usually reply within 1 day
-
Improve Type System for `pyperf`Possibly taken @DebanKsahu claimed this 69 days ago. Open
Difficulty 4/5 3-5 days Newbie friendliness 45/100
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 32/100
Maintainers usually reply within 1 day
-
Extend hook's teardown metadata with warmup/calibration flagsPossibly taken @maurycy claimed this 170 days ago. Open
Difficulty 4/5 3-5 days Newbie friendliness 48/100
psf/pyperf#239 · 12 comments ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
Maintainers usually reply within 1 day
Similar issues
-
plot_graph crashes when a mesh node has no m2m edgePossibly taken @tygvi claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
mllam/neural-lam#883 · 1 comment ·
Maintainers usually reply within 2 days
-
enhancement possible-duplicate
Difficulty 1/5 1-3 hours Newbie friendliness 78/100
AOSSIE-Org/PictoPy#1715 · 2 comments ·
Maintainers usually reply within 6 days
-
SQLite connector omits generated columns from the catalog, and drops relationships sourced on themOpen
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
NVIDIA/auto-ontology#511 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
ethereum/execution-specs#3996 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
cpp-best-practices/project_validator#267 ·
Maintainers usually reply within 1 day