Run LibreSign Behat shards in parallel using isolated Nextcloud environments

未关闭
#8,975 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
5/5
预计耗时
一周以上
新手友好度
35/100
Issue 类型
功能
描述清晰度
基本清楚
活跃度
活跃
技术栈
docker, github-actions, mariadb, mysql, php, postgresql, sqlite

调研方向

Start with the existing version-matrix workflow and the isolated nextcloud-docker-development worker lifecycle, then trace how Behat 4 shards and database entries are currently invoked. Benchmark 1, 2, and 4 workers with at least three successful runs per count; done means isolated concurrent workers, complete matrix coverage, diagnosable artifacts, equivalent results, and a documented reliable default.

由索引模型根据 Issue 内容生成。

描述

backend enhancement php

Part of #8969.

Context

The modernization work provides Behat 4, isolated tooling, concurrency-safe scenarios, deterministic sharding and isolated nextcloud-docker-development workers.

The suite can then be parallelized without sharing mutable Nextcloud state.

Goal

Execute multiple Behat shards concurrently and reduce integration-test feedback time without introducing cross-worker state leakage or unacceptable flakiness.

Execution model

Evaluate concurrent nextcloud-docker-development environments on the same CI runner before assuming that a separate GitHub Actions VM is required for every shard.

Each worker must receive a unique:

worker identifier
workspace/environment path
Compose project name
SHARD_INDEX
SHARD_TOTAL

Do not execute multiple shards against one shared Nextcloud installation.

CI security boundary

Parallel workers belonging to one test run may share an ephemeral runner and immutable caches/images.

Do not share mutable application state, credentials or worker directories between unrelated test runs.

For untrusted pull-request execution:

  • do not expose repository write credentials or long-lived secrets to test containers;
  • do not persist mutable Docker/Nextcloud state for reuse by a different PR;
  • cleanup must remove worker-specific containers/networks/state even after failures.

If self-hosted runners are used, the implementation must preserve an isolation boundary appropriate for untrusted PR code rather than assuming that Compose project names alone provide a security boundary.

Benchmark

Record the existing serial baseline before enabling the final configuration.

Compare at least where runner resources permit:

1 worker
2 workers
4 workers

Measure:

  • total wall-clock duration;
  • environment bootstrap time;
  • Behat execution time;
  • total runner usage;
  • CPU/memory pressure where practical;
  • failures/retries across repeated runs.

Select the default worker count from evidence rather than assuming that more workers are always faster.

Diagnostics

Every worker must produce identifiable output.

Where applicable, use worker/shard-specific artifacts such as:

report-shard-1.xml
report-shard-2.xml
nextcloud-shard-1.log

Enable useful Behat 4 failure reporting such as inline failures where it improves CI diagnosis.

A failure must clearly identify:

  • shard;
  • failing feature/scenario;
  • associated environment logs.

Reliability validation

Repeated parallel runs must include isolation-sensitive tests.

The implementation must not hide intermittent failures with unconditional retries.

If parallel execution reveals an existing race or scenario dependency, fix or isolate the cause rather than retrying until green.

Database matrix

Use the isolated nextcloud-docker-development worker lifecycle for the complete LibreSign Behat database matrix:

  • SQLite;
  • MySQL;
  • MariaDB 10.6;
  • MariaDB 10.11;
  • PostgreSQL.

Do not keep SQLite or MariaDB on a separate legacy execution architecture after the parallel migration is complete.

The matrix must continue to combine the database backend with the repository's supported PHP and Nextcloud versions according to the version-matrix workflow already used by LibreSign.

The implementation may choose different shard counts for different matrix entries when benchmark evidence justifies it, but it must not drop a database/PHP/Nextcloud compatibility entry merely to reduce CI time.

Benchmark protocol

Benchmark on the same runner class intended for the resulting CI workflow.

For each candidate worker count (1, 2 and 4 where resources permit):

  • perform at least three successful complete runs;
  • record bootstrap time and Behat execution time separately;
  • record total wall-clock time;
  • record the slowest shard duration;
  • record peak/observed resource pressure when practical;
  • record any failures or retries.

Use the median successful wall-clock result for the primary comparison, while retaining the individual measurements so outliers remain visible.

If four workers cannot complete reliably because of runner CPU/memory limits, record that result rather than hiding it and choose the highest reliable count.

The selected default worker count must be justified in the PR/implementation notes with these measurements.

Done when

  • Multiple isolated workers execute concurrently.
  • Every intended test is executed exactly once across the shards.
  • Serial and parallel results are behaviorally equivalent.
  • Parallel wall-clock performance is measured against baseline.
  • The selected default worker count is documented with benchmark evidence.
  • Worker-specific logs/reports make failures diagnosable.
  • Repeated parallel runs show no cross-worker state leakage.
  • Untrusted PR runs do not reuse mutable state or receive unnecessary long-lived/write credentials.
  • SQLite, MySQL, MariaDB 10.6/10.11 and PostgreSQL execute through the isolated worker lifecycle.
  • Existing PHP/Nextcloud/database compatibility coverage is preserved.
  • The selected concurrency level is supported by at least three measured successful runs on the target runner class.
主要语言
PHP
星标
844
派生
176
平均合并
14 小时 3 分钟
30 天内合并 PR
344

环境准备

在 Codespaces 中打开

在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

LibreSign/libresign 的其他 Issue

查看 LibreSign/libresign 的全部 Issue

相似的 Issue

更多 PHP Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。