Modernize LibreSign Behat infrastructure for Behat 4 and parallel execution

未关闭
#8,969 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
5/5
预计耗时
一周以上
新手友好度
20/100
Issue 类型
重构
描述清晰度
基本清楚
活跃度
活跃
技术栈
docker, php

调研方向

Treat this as an epic and begin by splitting the workstreams into focused child issues rather than implementing it as one change. Start with the shared package list, LibreSign's vendor-bin/behat toolchain, and nextcloud-docker-development; use the checklist as acceptance criteria, including Behat 4 execution, isolated deterministic parallel workers, preserved compatibility coverage, measured performance, and canonical documentation.

由索引模型根据 Issue 内容生成。

描述

enhancement

Context

LibreSign's Behat integration suite is a critical part of the project's regression coverage, but the current infrastructure still reflects assumptions from older Behat versions and executes each environment serially.

The current stack includes:

  • Behat 3;
  • YAML Behat configuration;
  • Behat dependencies installed under tests/integration;
  • shared Behat packages tied to Behat 3-era APIs;
  • reusable contexts coupled to older Gherkin argument objects;
  • PHPUnit used as a runtime assertion dependency in shared Behat code;
  • LibreSign feature/context definitions that have accumulated without a single modern Behat 4 convention;
  • a suite that is not explicitly designed for concurrent execution;
  • complete Behat runs that can take more than ten minutes.

Behat 4 provides the boundary for a coordinated modernization rather than a dependency-only upgrade.

The supported baseline for this work is:

  • PHP >= 8.2;
  • Behat ^4.0;
  • supported Nextcloud versions only.

The LibreSign migration applies to main only. It must not be backported to existing stable branches.

Goal

Modernize the complete LibreSign Behat testing stack around Behat 4 and make the suite safe, maintainable and measurably faster through isolated parallel execution.

Workstreams

Shared Behat packages

Modernize and publish Behat 4-compatible breaking releases of the packages LibreSign depends on:

  • libresign/behat-builtin-extension;
  • libresign/nextcloud-behat;
  • libresign/mailpit-behat-extension;
  • libresign/behat-tsa-extension.

The shared package work also includes:

  • removing PHPUnit as a runtime assertion implementation from nextcloud-behat;
  • adopting DataTable and DocString;
  • auditing reusable generic steps for Behat 4 signatures and clearer contracts;
  • relying on Behat public APIs rather than unsupported internals.
LibreSign Behat toolchain

LibreSign should:

  • install Behat through vendor-bin/behat;
  • use PHP configuration instead of Behat YAML;
  • consume released Behat 4-compatible shared packages;
  • keep the Behat Composer environment resolvable for PHP 8.2;
  • prefer Behat's built-in JUnit support when it satisfies the existing CI reporting contract.
Test design

Define and apply:

  • a small semantic tag taxonomy;
  • ownership boundaries between generic Nextcloud steps and LibreSign domain steps;
  • modern Behat 4 context/feature conventions;
  • independent scenario setup and lifecycle rules.

Tags describe test semantics. They must not be manually maintained shard identifiers.

Parallel execution

Use LibreCodeCoop/nextcloud-docker-development as the isolated Nextcloud environment for the complete parallel Behat database matrix. The worker lifecycle must support SQLite, MySQL, MariaDB 10.6/10.11 and PostgreSQL so the migration does not leave part of the compatibility matrix on a separate legacy execution architecture.

Each worker must have isolated mutable state, including where applicable:

  • Nextcloud installation/configuration;
  • database;
  • data directory;
  • users/groups;
  • app configuration;
  • generated files;
  • mail state;
  • mock/local services;
  • temporary files;
  • reports/logs.

Sharding must be deterministic and independent from semantic tagging.

Architecture

Shared package modernization
├── behat-builtin-extension
├── nextcloud-behat
│   ├── standalone assertions
│   ├── DataTable / DocString
│   └── reusable step modernization
├── mailpit-behat-extension
└── behat-tsa-extension
        ↓
LibreSign vendor-bin/behat + Behat 4
        ↓
LibreSign context/feature modernization

Test taxonomy ────────────────┐
Scenario concurrency safety ─┤
Deterministic sharding ──────┼─> isolated parallel Behat execution
Docker worker lifecycle ─────┘
        ↓
final Behat 3 cleanup + canonical documentation

Test-design principles

Scenarios must:

  • be executable independently where intended for normal parallel execution;
  • not rely on execution order;
  • create or declare the state they require;
  • not depend on state left by another scenario;
  • avoid unnecessary implementation-detail steps;
  • reuse generic Nextcloud steps when behavior is generic;
  • keep LibreSign business-language steps in the LibreSign layer.

Parallel-execution principles

Parallelism must be introduced only after isolation prerequisites are proven.

Parallel workers belonging to one test run may share immutable caches/images, but must not share mutable application state.

Untrusted pull-request runs must not receive unnecessary repository write credentials or long-lived secrets, and mutable state must not be reused across unrelated test runs.

Performance

Record a serial baseline before selecting the final concurrency level.

Compare at least, where runner resources permit:

  • one worker;
  • two workers;
  • four workers.

Measure:

  • wall-clock duration;
  • environment bootstrap duration;
  • Behat execution duration;
  • CPU/memory pressure where practical;
  • total runner usage;
  • failures/retries;
  • evidence of flakiness or state leakage.

Do not select the final worker count without measurement.

Compatibility

  • Target PHP >= 8.2.
  • Target Behat ^4.0.
  • New breaking releases of the shared testing packages do not retain Behat 3 compatibility.
  • LibreSign changes target main.
  • Do not backport this migration to stable branches.

Done when

  • All required shared Behat packages have Composer-installable Behat 4-compatible breaking releases.
  • LibreSign main executes its integration suite using Behat 4.
  • Behat tooling is isolated under vendor-bin/behat and remains resolvable for PHP 8.2.
  • YAML Behat configuration is no longer required.
  • Shared contexts use supported/public Behat APIs.
  • PHPUnit is not used merely as the runtime assertion library of shared Behat contexts.
  • Reusable step APIs use modern Behat argument types where applicable.
  • Generic Nextcloud reusable steps and LibreSign domain steps have documented ownership boundaries and have been modernized accordingly.
  • The test taxonomy is documented and applied.
  • Scenarios intended for normal parallel execution are independent of execution order and shared mutable state.
  • nextcloud-docker-development supports the complete Behat database matrix: SQLite, MySQL, MariaDB 10.6/10.11 and PostgreSQL.
  • Multiple isolated Nextcloud test environments can run concurrently.
  • Deterministic sharding executes every intended test exactly once across workers.
  • Parallel CI is measurably faster than the serial baseline.
  • Parallel execution does not introduce measurable cross-worker state leakage or systematic flakiness.
  • Failure output identifies the failing worker/shard and remains diagnosable.
  • Existing database/Nextcloud/PHP compatibility coverage is preserved.
  • Legacy Behat 3 infrastructure is removed after the new path is proven.
  • The supported developer workflow is documented in LibreSign's canonical documentation.

Target

Branch: main

Milestone: Next Major (36)

Pull requests should implement focused child issues, not this epic.

主要语言
PHP
星标
844
派生
176
平均合并
14 小时 3 分钟
30 天内合并 PR
344

环境准备

在 Codespaces 中打开

在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

LibreSign/libresign 的其他 Issue

查看 LibreSign/libresign 的全部 Issue

相似的 Issue

更多 PHP Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。