webdriverio/webdriverio

Cache list of spec files

Open

#4,880 opened on Dec 14, 2019

View on GitHub
 (2 comments) (0 reactions) (0 assignees)JavaScript (1,793 forks)batch import
Performancehelp wanted

Repository metrics

Stars
 (6,029 stars)
PR merge metrics
 (Avg merge 15d 15h) (48 merged PRs in 30d)

Description

Is your feature request related to a problem? Please describe. Finding specs in a big monorepo takes ages. Think 130K+ files (sources + tests + assets). Rough directory structure is this:

page1/
	- features/
		page1.feature
	- subpage1/
		- features/
			- subpage1.feature
		- subpage1.js
	- page1.js
	- page1.css
page2/
	<same structure>
ui/shared_component/
	- features/
		- shared_component_in_page1.feature
		- shared_component_in_page2.feature
	- shared_component.js

This is a screenshot from a JS CPU profile of the first ~2 minutes of running WDIO on our repo. At that point, it didn't even get to actually running the tests.

Note how consistently ConfigParser#getSpecs appears in the chart, and how much time it consumes.

This massive time is to find just ~250 .feature files among the sources. I also had passed a tag expression that would filter that down to just 4 files, so it's unacceptable that WDIO takes this long.

Describe the solution you'd like ConfigParser#getSpecs could be memoized, so that when it takes the same input, it always returns the same value.

I guess that adding/removing files could become a problem with that solution, in which case it should be coupled with a file system watching solution to manipulate the cache.

Describe alternatives you've considered The specs option was too inclusive (src/**/features/*.feature), so I tinkered with the suites option:

suites: {
	page1: ['src/page1/**/features/*.feature', 'src/ui/**/features/*.feature']
	// and so on
}

This didn't help bringing the time to boot up down noticeably, though.

And, as probably obvious from above, keeping a list of .feature files in the config that doesn't make use of wildcards is impossible.

Additional context Node.js v10.16.3

@wdio packages used:

    "@wdio/cli": "^5.16.10",
    "@wdio/cucumber-framework": "^5.16.10",
    "@wdio/local-runner": "^5.16.10",
    "@wdio/selenium-standalone-service": "^5.16.10",
    "@wdio/spec-reporter": "^5.16.10",
    "@wdio/sync": "^5.16.10",

Contributor guide