Run Behat in throwaway containers when Docker is available
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 45/100
Research direction
Read #390 to see the worker design and #384 for the baseline timing. The repo likely needs a Dockerfile for a tuned PHP image without Xdebug, and changes to the test runner to launch throwaway containers per worker. Done when a draft PR runs CI jobs with containerized Behat and shows snapshot restore working. Fallback to regular runner must remain when Docker is absent.
Written by the indexing model from the issue text.
Description
When Docker is available, a run could happen in throwaway containers, one per worker from #390. The container is not faster by itself: PHP ran at the same speed inside and outside of it in local tests. What it allows without root on the host is what makes the difference:
- the temporary directories and the database in memory, with no disk syncs;
- copy-on-write snapshots of the cached WordPress install instead of a full copy for every scenario (about 6 ms instead of about 90 ms to set up and tear down);
- a separate network per worker, so scenarios using fixed ports can run in parallel;
- a known toolchain, so every declared test requirement is present.
Starting a container takes about 0.3 to 0.5 seconds and every docker exec about 50 to 75 ms. Running the whole Behat process of a worker inside one container avoids that cost per command. The container's PHP version has to match the one the vendor folder was installed for.
Steps:
- Provide an image tuned for the test runs, without Xdebug unless coverage is requested.
- Run each worker of #390 in its own throwaway container, falling back to the regular runner when Docker is not available.
- Restore the cached WordPress install as a snapshot instead of a copy.
To reproduce a failure, it is enough to rerun that single scenario in a container that is kept.
Measuring: take the baseline from #384, then open a draft PR, run the same CI jobs against it and record both results here before we decide.
- Dominant language
- PHP
- Stars
- 42
- Forks
- 28
- Avg merge
- 17h 14m
- Merged PRs (30d)
- 11
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from wp-cli/wp-cli-tests
-
scope:testing status:unconfirmed
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
wp-cli/wp-cli-tests#389 · 1 comment ·
Maintainers usually reply within 1 day
-
scope:testing
Difficulty 5/5 Over a week Newbie friendliness 20/100
wp-cli/wp-cli-tests#395 ·
Maintainers usually reply within 1 day
-
scope:testing
Difficulty 4/5 3-5 days Newbie friendliness 30/100
wp-cli/wp-cli-tests#394 ·
Maintainers usually reply within 1 day
-
Declare and check the external tools that scenarios needPossibly taken A pull request linked to this issue is open or already merged. Openbug scope:testing
Difficulty 4/5 3-5 days Newbie friendliness 45/100
wp-cli/wp-cli-tests#393 ·
Maintainers usually reply within 1 day
-
Report scenarios that are filtered out by unmet requirementsPossibly taken A pull request linked to this issue is open or already merged. Openscope:testing
Difficulty 3/5 1-2 days Newbie friendliness 70/100
wp-cli/wp-cli-tests#392 ·
Maintainers usually reply within 1 day
All issues in wp-cli/wp-cli-tests
Similar issues
-
enhancement
Difficulty 1/5 Under an hour Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
grokability/snipe-it#19996 ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 1/5 Under an hour Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
Add a dark themeOpengood first issue laravel-appointment-reminders
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Text to Speech fails on Windows: the temp file path loses its backslashes when stored in post metaPossibly taken @azizulhasan claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day