taichi-dev/taichi

[benchmark] Use pytest-benchmark for performance regression test

Open

#1,454 opened on Jul 10, 2020

View on GitHub
 (1 comment) (2 reactions) (1 assignee)C++ (2,384 forks)batch import
feature requestgood first issuepython

Repository metrics

Stars
 (28,195 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

Concisely describe the proposed feature I'd like to make use of the pytest plugin pytest-benchmark instead of our own ad-hoc benchmarking framework which I've done with.

Describe the solution you'd like (if any) First check out the benchmarks/ directory, these are the benchmarks that we've already have. Make function names like benchmark_ -> test_. And make use of that plugin, as describe in their project page. When runing benchmark, simply use pytest benchmarks/. You may change the behavior ti benchmark to follow the same logic as ti test does.

Additional comments

Contributor guide

Research direction
Review the existing benchmarks in the benchmarks/ directory and understand the pytest benchmark plugin usage. Modify benchmark functions to follow pytest benchmark conventions and update the ti benchmark command.
Tech stack
python
Domain
testing
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Mostly clear
Prerequisites
Pythonpytest
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
60