Pragmatists/JUnitParams

JUnit fails to report results of some tests if used with surefire and parallel option including methods

Open

#35 aberto em 18 de fev. de 2015

Ver no GitHub
 (3 comments) (0 reactions) (0 assignees)HTML (151 forks)batch import
bughelp wanted

Métricas do repositório

Stars
 (887 stars)
Métricas de merge de PR
 (Nenhuma PRs mesclada em 30d)

Description

Example: run mvn test in JUnitParams project with following configuration:

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.18.1</version>

                <configuration>
                    <runOrder>filesystem</runOrder>
                    <parallel>classesAndMethods</parallel>
                    <perCoreThreadCount>false</perCoreThreadCount>
                    <threadCount>16</threadCount>
                    <useUnlimitedThreads>false</useUnlimitedThreads>
                    <threadCountSuites>0</threadCountSuites>
                    <threadCountClasses>0</threadCountClasses>
                    <threadCountMethods>0</threadCountMethods>
                    <parallelOptimized>true</parallelOptimized>
                    <includes>
                        <include>**/SubclassTest.java</include>
                        <include>**/Samples_*</include>
                    </includes>
                </configuration>
            </plugin>

This should give total of 44 tests as of today, but often it will give less than that. Contact me for details about research done so far. Suspicious places:

  • org.apache.maven.surefire.junitcore.JUnitCoreRunListener#fillTestCountMap
  • org.apache.maven.surefire.junitcore.ClassesParallelRunListener#checkIfTestSetCanBeReported
  • org.apache.maven.surefire.junitcore.TestSet#setAllScheduled

Currently following values of surefire "parallel" option should not be used:

  • "classesAndMethods"
  • "methods"
  • "both"

Guia do colaborador