[SR-1266] Fix duplication of @VARIABLES@ in test and validation-test lit.site.cfg.in

Previous ID	SR-1266
Radar	None
Original Reporter	@modocache
Type	Bug

Votes	0
Component/s	Project Infrastructure
Labels	Bug, CMake, StarterBug
Assignee	None
Priority	Medium

md5: 415295060b366f5b63750d3f4a872cf4

Issue Description:

The test and validation-test directories in the Swift project define tests and validation tests. The lit testing tool is used to run the tests. lit allows test suites to be configured via a file named lit.cfg.

Both test/lit.cfg and validation-test/lit.cfg use parameters that are defined as part of the Swift CMake build scripts. The CMake portion of the build system is aware of a lot of information that is useful to the tests, such as the location where LLVM build products will be placed--something like /path/to/build/llvm-macosx-x86_64/bin. CMake communicates these values to lit via the files test/lit.site.cfg.in and validation-test/lit.site.cfg.in. You can see the exact lines that CMake find-and-replaces variables like @LLVM_BINARY_DIR@ here.

I was recently bit by this system: I had added a new variable, @SWIFT_ANDROID_NDK_PATH@, to test/lit.site.cfg.in, but not to validation-test/lit.site.cfg.in. I was confused why my tests failed to be configured with the correct path sometimes.

It would be nice if I only had to add variables like @SWIFT_ANDROID_NDK_PATH@ to a single file, as opposed to several.

Accomplishing this requires basic knowledge of how the Swift test suite is run, as well some CMake and Python syntax. This task is a good opportunity to learn more about the Swift build system, its test suite, CMake, and lit.