Instruction counts instead of wall clock time?
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- linux, python
- Domain
- performance, tooling
Research direction
The issue names no files or tests. Start by investigating how pyperf records wall-clock measurements and how Linux's perf module and hardware performance counters could be accessed, then compare whether instruction counts provide a useful alternative across supported platforms.
Written by the indexing model from the issue text.
Description
It would be interesting to investigate use of instruction counts (through Linux's perf module and similar tools on other platforms to access hardware performance counters) within pyperf.
See, for example, Nicholas Nethercote's experience with monitoring rustc performance:
Contrary to what you might expect, instruction counts have proven much better than wall times when it comes to detecting performance changes on CI, because instruction counts are much less variable than wall times (e.g. ±0.1% vs ±3%; the former is highly useful, the latter is barely useful). Using instruction counts to compare the performance of two entirely different programs (e.g. GCC vs clang) would be foolish, but it’s reasonable to use them to compare the performance of two almost-identical programs (e.g. rustc before PR #12345 and rustc after PR #12345). It’s rare for instruction count changes to not match wall time changes in that situation. If the parallel version of the rustc front-end ever becomes the default, it will be interesting to see if instruction counts continue to be effective in this manner.
Perhaps in an interpreter where dispatch overhead and boxing/unboxing cost can be significant this won't hold true due to small changes having the potential to cause to a much more significant change in cache misses, but it would still be worthwhile to investigate in my view.
- Dominant language
- Python
- Stars
- 967
- Forks
- 112
- Avg merge
- 5d 8h
- Merged PRs (30d)
- 9
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from psf/pyperf
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
Maintainers usually reply within 1 day
-
Improve Type System for `pyperf`Possibly taken @DebanKsahu claimed this 69 days ago. Open
Difficulty 4/5 3-5 days Newbie friendliness 45/100
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 32/100
Maintainers usually reply within 1 day
-
Extend hook's teardown metadata with warmup/calibration flagsPossibly taken @maurycy claimed this 170 days ago. Open
Difficulty 4/5 3-5 days Newbie friendliness 48/100
psf/pyperf#239 · 12 comments ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
Maintainers usually reply within 1 day
Similar issues
-
plot_graph crashes when a mesh node has no m2m edgePossibly taken @tygvi claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
mllam/neural-lam#883 · 1 comment ·
Maintainers usually reply within 2 days
-
enhancement possible-duplicate
Difficulty 1/5 1-3 hours Newbie friendliness 78/100
AOSSIE-Org/PictoPy#1715 · 2 comments ·
Maintainers usually reply within 6 days
-
SQLite connector omits generated columns from the catalog, and drops relationships sourced on themOpen
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
NVIDIA/auto-ontology#511 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
ethereum/execution-specs#3996 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
cpp-best-practices/project_validator#267 ·
Maintainers usually reply within 1 day