Base manifest doesn't allow benchmark groups except "default" benchmark group
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 38/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- python
- Domain
- performance
Research direction
Start by reproducing the issue with python3 -m venv testvenv, installing pyperformance, and running testvenv/bin/pyperformance run -b math. Trace the pyperformance run entry point and base-manifest handling to find why only the default benchmark group is accepted; done means the math group runs its three benchmarks instead of reporting that no benchmark was run.
Written by the indexing model from the issue text.
Description
Where applicable, please fill out the details below to help us troubleshoot the
issue that you are facing. 🍰
How to reproduce the issue
Using script:
python3 -m venv testvenv
source testvenv/bin/activate
pip install pyperformance
testvenv/bin/pyperformance run -b math
What you expected to happen
The tool will run all the math benchmarks producing output like:
[1/3] float...
.....................
float: Mean +- std dev: 48.3 ms +- 1.6 ms
[2/3] nbody...
.....................
nbody: Mean +- std dev: 46.6 ms +- 1.7 ms
[3/3] pidigits...
What actually happens
Collecting pyperformance
Using cached pyperformance-1.11.0-py3-none-any.whl.metadata (2.0 kB)
Collecting pyperf (from pyperformance)
Using cached pyperf-2.7.0-py3-none-any.whl.metadata (6.0 kB)
Collecting packaging (from pyperformance)
Using cached packaging-24.1-py3-none-any.whl.metadata (3.2 kB)
Collecting psutil>=5.9.0 (from pyperf->pyperformance)
Using cached psutil-6.0.0-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (21 kB)
Using cached pyperformance-1.11.0-py3-none-any.whl (4.2 MB)
Using cached packaging-24.1-py3-none-any.whl (53 kB)
Using cached pyperf-2.7.0-py3-none-any.whl (139 kB)
Using cached psutil-6.0.0-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (290 kB)
Installing collected packages: psutil, packaging, pyperf, pyperformance
Successfully installed packaging-24.1 psutil-6.0.0 pyperf-2.7.0 pyperformance-1.11.0
WARNING:root:no benchmark named 'math'
Python benchmark suite 1.11.0
ERROR: No benchmark was run
- Dominant language
- Python
- Stars
- 1k
- Forks
- 203
- Avg merge
- 1h 20m
- Merged PRs (30d)
- 2
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from python/pyperformance
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
python/pyperformance#346 · 3 comments ·
-
python/pyperformance#474 · 4 reactions · 1 assignee ·
-
Difficulty 3/5 1-2 days Newbie friendliness 25/100
python/pyperformance#473 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 55/100
python/pyperformance#464 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
python/pyperformance#460 · 2 comments · 1 reaction ·
All issues in python/pyperformance
Similar issues
-
Add: hunch Open
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
AbdelStark/awesome-typesafe#104 ·
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
DiamondLightSource/dodal#2211 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
openml/openml-python#1749 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
sipyourdrink-ltd/bernstein#6191 ·