webdriverio/webdriverio

Cache list of spec files

Open

#4,880 opened on Dec 14, 2019

View on GitHub
 (2 comments) (0 reactions) (0 assignees)JavaScript (6,029 stars) (1,793 forks)batch import
Performancehelp wanted

Description

Is your feature request related to a problem? Please describe. Finding specs in a big monorepo takes ages. Think 130K+ files (sources + tests + assets). Rough directory structure is this:

page1/
	- features/
		page1.feature
	- subpage1/
		- features/
			- subpage1.feature
		- subpage1.js
	- page1.js
	- page1.css
page2/
	<same structure>
ui/shared_component/
	- features/
		- shared_component_in_page1.feature
		- shared_component_in_page2.feature
	- shared_component.js

This is a screenshot from a JS CPU profile of the first ~2 minutes of running WDIO on our repo. At that point, it didn't even get to actually running the tests.

Note how consistently ConfigParser#getSpecs appears in the chart, and how much time it consumes.

This massive time is to find just ~250 .feature files among the sources. I also had passed a tag expression that would filter that down to just 4 files, so it's unacceptable that WDIO takes this long.

Describe the solution you'd like ConfigParser#getSpecs could be memoized, so that when it takes the same input, it always returns the same value.

I guess that adding/removing files could become a problem with that solution, in which case it should be coupled with a file system watching solution to manipulate the cache.

Describe alternatives you've considered The specs option was too inclusive (src/**/features/*.feature), so I tinkered with the suites option:

suites: {
	page1: ['src/page1/**/features/*.feature', 'src/ui/**/features/*.feature']
	// and so on
}

This didn't help bringing the time to boot up down noticeably, though.

And, as probably obvious from above, keeping a list of .feature files in the config that doesn't make use of wildcards is impossible.

Additional context Node.js v10.16.3

@wdio packages used:

    "@wdio/cli": "^5.16.10",
    "@wdio/cucumber-framework": "^5.16.10",
    "@wdio/local-runner": "^5.16.10",
    "@wdio/selenium-standalone-service": "^5.16.10",
    "@wdio/spec-reporter": "^5.16.10",
    "@wdio/sync": "^5.16.10",

Contributor guide