Waiting for runner's bench_func and bench_command functions to complete instead of receiving outputs individually?

Open
  • Is something wrong?
#141 5 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Stale
Tech stack
python
Domain
performance

Research direction

Start with the documented pyperf.Runner and bench_func example, then inspect how benchmark results and console output are exposed. Compare the requested multiprocessing and executor use cases with the current runner behavior. Done means the issue's desired result collection and completion behavior are clearly supported or documented, with the unwanted intermediate output addressed.

Written by the indexing model from the issue text.

Description

I'm trying to utilize pyperf to benchmark some functions and save their results to a CSV of my formatting.
I'm using this example code from the documentation:

#!/usr/bin/env python3
import pyperf
import time


def func():
    time.sleep(0.001)

runner = pyperf.Runner()
runner.bench_func('sleep', func)

I want to benchmark the same function with mulitprocessing.pool.Pool / mulitprocessing.pool.ThreadPool / concurrent.futures.ProcessPoolExecutor / concurrent.futures.ThreadPoolExecutor and varying values for things like the number of CPU cores and the chunksize for map functions.

The issue is that assigning a variable to store the output of runner.bench_func and printing out that variable leads to an output like this:

<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.
python_startup: Mean +- std dev: 24.5 ms +- 0.4 ms
<Benchmark 'python_startup' with 21 runs>

Whereas I want to suppress this output and wait for all runs to complete before moving forward with the program.

Is there some other way of storing the results of a benchmark that I can't seem to find in the documentation? Or should is there a way to force the runner to wait for a benchmark to complete?

Dominant language
Python
Stars
967
Forks
112
Avg merge
5d 8h
Merged PRs (30d)
9

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from psf/pyperf

All issues in psf/pyperf

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.