Skip the Docker containers for suites too small to gain from them

未关闭
  • 哪里不对？
#426 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
4/5
预计耗时
3-5 天
新手友好度
22/100
Issue 类型
功能
描述清晰度
基本清楚
活跃度
活跃
技术栈
docker, mysql, php
领域
testing-qa

调研方向

The decision lives in utils/behat-parallel.php, which already computes the per-scenario cost used to balance workers; start there and check how the Docker fallback is chosen when WP_CLI_TEST_DOCKER=auto. The Docker image is built from utils/docker/Dockerfile, which #425 covers. Done means a threshold that skips containers for the small packages listed in the issue, a fallback message like the existing ones, and matching timings on the full matrix.

由索引模型根据 Issue 内容生成。

描述

scope:testing

Part of #394.

Since wp-cli/.github#313, every Linux Behat job runs its scenarios in 4 parallel workers in Docker containers. Large suites gain a lot from it. Small suites pay more than they get back, because every job has a fixed setup cost of about 70 to 80 seconds, whatever its size:

  • 40 to 63 seconds before the first worker starts. Most of this is building the worker image from utils/docker/Dockerfile (see #425 for that part).
  • Another 20 to 30 seconds for starting the containers and database servers, and for removing them again.

In the full-matrix runs of all 29 packages on 2026-11-02, compared with the median of their October runs on main, runner time per Testing run changed as follows:

Packages Before (min) After (min) Change
Large suites 9 3,762 1,893 −50%
Small suites 20 1,734 2,214 +28%

In the job for PHP 8.4, WP latest and MySQL 8.0:

Package Scenarios Behat step Longest worker
maintenance-mode-command 3 71 s 10 s
role-command 5 84 s 14 s
site-health-command 5 87 s 16 s
super-admin-command 5 91 s 20 s
eval-command 21 91 s 13 s
entity-command, for comparison 511 457 s 372 s

Before the change, maintenance-mode-command used 49 runner-minutes per Testing run; now it uses 97.

Proposal: with WP_CLI_TEST_DOCKER=auto, run the scenarios without Docker when their expected cost is too low for the containers to pay off, and report it like the other fallbacks ("... Running the scenarios without Docker."). The expected cost per scenario that utils/behat-parallel.php already computes for balancing the workers can decide this, so the threshold does not depend on scenario counts alone. Below the threshold, parallel workers on the runner itself (#405) are still available. These add only a few seconds, as long as the test user can create the worker databases.

WP_CLI_TEST_DOCKER=1 keeps forcing containers, so that a small suite can still be run in them deliberately.

Measuring: pick the threshold from the per-job numbers above. Then run the full matrix of a few small packages (for example maintenance-mode-command, role-command, eval-command, cron-command) and a few packages near the threshold (for example cache-command, export-command) against the change, and compare runner time and Behat time per job with the runs of 2026-11-02. If #425 lands first, measure against it, as a cheaper image moves the break-even point.

主要语言
PHP
星标
42
派生
28
平均合并
18 小时 52 分钟
30 天内合并 PR
27

环境准备

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

wp-cli/wp-cli-tests 的其他 Issue

查看 wp-cli/wp-cli-tests 的全部 Issue

相似的 Issue

更多 PHP Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。