Pragmatists/JUnitParams

JUnit fails to report results of some tests if used with surefire and parallel option including methods

Open

#35 opened on Feb 18, 2015

View on GitHub
 (3 comments) (0 reactions) (0 assignees)HTML (151 forks)batch import
bughelp wanted

Repository metrics

Stars
 (887 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

Example: run mvn test in JUnitParams project with following configuration:

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.18.1</version>

                <configuration>
                    <runOrder>filesystem</runOrder>
                    <parallel>classesAndMethods</parallel>
                    <perCoreThreadCount>false</perCoreThreadCount>
                    <threadCount>16</threadCount>
                    <useUnlimitedThreads>false</useUnlimitedThreads>
                    <threadCountSuites>0</threadCountSuites>
                    <threadCountClasses>0</threadCountClasses>
                    <threadCountMethods>0</threadCountMethods>
                    <parallelOptimized>true</parallelOptimized>
                    <includes>
                        <include>**/SubclassTest.java</include>
                        <include>**/Samples_*</include>
                    </includes>
                </configuration>
            </plugin>

This should give total of 44 tests as of today, but often it will give less than that. Contact me for details about research done so far. Suspicious places:

  • org.apache.maven.surefire.junitcore.JUnitCoreRunListener#fillTestCountMap
  • org.apache.maven.surefire.junitcore.ClassesParallelRunListener#checkIfTestSetCanBeReported
  • org.apache.maven.surefire.junitcore.TestSet#setAllScheduled

Currently following values of surefire "parallel" option should not be used:

  • "classesAndMethods"
  • "methods"
  • "both"

Contributor guide