Pragmatists/JUnitParams

JUnit fails to report results of some tests if used with surefire and parallel option including methods

Open

#35 opened on Feb 18, 2015

View on GitHub
 (3 comments) (0 reactions) (0 assignees)HTML (887 stars) (151 forks)batch import
bughelp wanted

Description

Example: run mvn test in JUnitParams project with following configuration:

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.18.1</version>

                <configuration>
                    <runOrder>filesystem</runOrder>
                    <parallel>classesAndMethods</parallel>
                    <perCoreThreadCount>false</perCoreThreadCount>
                    <threadCount>16</threadCount>
                    <useUnlimitedThreads>false</useUnlimitedThreads>
                    <threadCountSuites>0</threadCountSuites>
                    <threadCountClasses>0</threadCountClasses>
                    <threadCountMethods>0</threadCountMethods>
                    <parallelOptimized>true</parallelOptimized>
                    <includes>
                        <include>**/SubclassTest.java</include>
                        <include>**/Samples_*</include>
                    </includes>
                </configuration>
            </plugin>

This should give total of 44 tests as of today, but often it will give less than that. Contact me for details about research done so far. Suspicious places:

  • org.apache.maven.surefire.junitcore.JUnitCoreRunListener#fillTestCountMap
  • org.apache.maven.surefire.junitcore.ClassesParallelRunListener#checkIfTestSetCanBeReported
  • org.apache.maven.surefire.junitcore.TestSet#setAllScheduled

Currently following values of surefire "parallel" option should not be used:

  • "classesAndMethods"
  • "methods"
  • "both"

Contributor guide