AccelerateHS/accelerate

Automated regression testing

Open

#214 opened on Oct 30, 2014

View on GitHub
 (3 comments) (0 reactions) (0 assignees)Haskell (830 stars) (112 forks)batch import
help wanted

Description

We need to set up some kind of performance regression testing.

@rrnewton and I started to set this up at http://codespeed.crest.iu.edu, following @nomeata's example for ghc at http://ghcspeed-nomeata.rhcloud.com, but so far we haven't managed to get it to work. We are not pythonistas though, so if anybody wants to volunteer to help, and perhaps sway my opinion of untyped languages in the process, please do.

I'll leave this open to remind us to sort it out.

Contributor guide

Tech stack
None
Domain
testingperformancedevops
Issue type
devops
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
4
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
over 1 week
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
mostly clear
Prerequisites
HaskellPython basicsperformance testingcodespeed
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
20
Research direction
Check the existing codespeed instance at http://codespeed.crest.iu.edu and the GHC example at http://ghcspeed nomeata.rhcloud.com. Investigate why the initial setup failed and identify the missing steps. Reach out to @rrnewton and @nomeata for guidance on configuring the tool for Haskell benchmarks. The deliverable is a working performance regression dashboard and a guide to reproduce it.