Pragmatists/JUnitParams

JUnit fails to report results of some tests if used with surefire and parallel option including methods

Open

#35 aperta il 18 feb 2015

Vedi su GitHub
 (3 commenti) (0 reazioni) (0 assegnatari)HTML (151 fork)batch import
bughelp wanted

Metriche repository

Star
 (887 star)
Metriche merge PR
 (Nessuna PR mergiata in 30 g)

Descrizione

Example: run mvn test in JUnitParams project with following configuration:

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.18.1</version>

                <configuration>
                    <runOrder>filesystem</runOrder>
                    <parallel>classesAndMethods</parallel>
                    <perCoreThreadCount>false</perCoreThreadCount>
                    <threadCount>16</threadCount>
                    <useUnlimitedThreads>false</useUnlimitedThreads>
                    <threadCountSuites>0</threadCountSuites>
                    <threadCountClasses>0</threadCountClasses>
                    <threadCountMethods>0</threadCountMethods>
                    <parallelOptimized>true</parallelOptimized>
                    <includes>
                        <include>**/SubclassTest.java</include>
                        <include>**/Samples_*</include>
                    </includes>
                </configuration>
            </plugin>

This should give total of 44 tests as of today, but often it will give less than that. Contact me for details about research done so far. Suspicious places:

  • org.apache.maven.surefire.junitcore.JUnitCoreRunListener#fillTestCountMap
  • org.apache.maven.surefire.junitcore.ClassesParallelRunListener#checkIfTestSetCanBeReported
  • org.apache.maven.surefire.junitcore.TestSet#setAllScheduled

Currently following values of surefire "parallel" option should not be used:

  • "classesAndMethods"
  • "methods"
  • "both"

Guida contributor