ollef/sixten

Generator and simplifier for property based tests

Open

#67 opened on Dec 3, 2017

View on GitHub
 (1 comment) (0 reactions) (0 assignees)Haskell (631 stars) (24 forks)batch import
Help wantedType: Feature

Description

In #14, you suggested that we should have property based testing of the simplifier.

  • I think the same generator will be useful for a variety of other tests later on (e.g. an integration test from simplification to code gen), so it would be very useful to have a good shrinker with it.
  • I saw this fun ICFP talk [paper] about compiler testing using code generation. I don't know if this is directly helpful but maybe their citations help in writing a good generator or shrinker. While many of the examples given rely on being able to perform IO arbitrarily, some like (/) 0 (let e = not in pred 1) don't depend on this. It might be a good test to check that the output is identical for different optimization levels etc.

Contributor guide

Tech stack
None
Domain
testing
Issue type
test
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Haskell programmingProperty based testing conceptssixten simplifier knowledge
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
30
Research direction
Review issue #14 which discusses property based testing of the simplifier. Look at the sixten codebase to understand the data types used in the simplifier and existing test structure. Design a generator and shrinker for sixten programs based on the referenced ICFP talk and paper. Implement the generator and shrinker in Haskell, then create property based tests that verify simplifier correctness. Consider integrating with a property testing library like QuickCheck.