Modernize LibreSign Behat infrastructure for Behat 4 and parallel execution
Maintainer thường phản hồi trong vòng 1 ngày
Chưa có ai nhận issue này.
Đánh giá
- Độ khó
- 5/5
- Thời gian dự kiến
- Hơn một tuần
- Mức phù hợp với người mới
- 20/100
- Loại issue
- Tái cấu trúc
- Độ rõ ràng
- Khá rõ ràng
- Mức độ hoạt động
- Sôi nổi
- Công nghệ
- docker, php
- Lĩnh vực
- ci-cd, devops, documentation, testing-qa, tooling
Hướng nghiên cứu
Treat this as an epic and begin by splitting the workstreams into focused child issues rather than implementing it as one change. Start with the shared package list, LibreSign's vendor-bin/behat toolchain, and nextcloud-docker-development; use the checklist as acceptance criteria, including Behat 4 execution, isolated deterministic parallel workers, preserved compatibility coverage, measured performance, and canonical documentation.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Mô tả
Context
LibreSign's Behat integration suite is a critical part of the project's regression coverage, but the current infrastructure still reflects assumptions from older Behat versions and executes each environment serially.
The current stack includes:
- Behat 3;
- YAML Behat configuration;
- Behat dependencies installed under
tests/integration; - shared Behat packages tied to Behat 3-era APIs;
- reusable contexts coupled to older Gherkin argument objects;
- PHPUnit used as a runtime assertion dependency in shared Behat code;
- LibreSign feature/context definitions that have accumulated without a single modern Behat 4 convention;
- a suite that is not explicitly designed for concurrent execution;
- complete Behat runs that can take more than ten minutes.
Behat 4 provides the boundary for a coordinated modernization rather than a dependency-only upgrade.
The supported baseline for this work is:
- PHP >= 8.2;
- Behat ^4.0;
- supported Nextcloud versions only.
The LibreSign migration applies to main only. It must not be backported to existing stable branches.
Goal
Modernize the complete LibreSign Behat testing stack around Behat 4 and make the suite safe, maintainable and measurably faster through isolated parallel execution.
Workstreams
Shared Behat packages
Modernize and publish Behat 4-compatible breaking releases of the packages LibreSign depends on:
libresign/behat-builtin-extension;libresign/nextcloud-behat;libresign/mailpit-behat-extension;libresign/behat-tsa-extension.
The shared package work also includes:
- removing PHPUnit as a runtime assertion implementation from
nextcloud-behat; - adopting
DataTableandDocString; - auditing reusable generic steps for Behat 4 signatures and clearer contracts;
- relying on Behat public APIs rather than unsupported internals.
LibreSign Behat toolchain
LibreSign should:
- install Behat through
vendor-bin/behat; - use PHP configuration instead of Behat YAML;
- consume released Behat 4-compatible shared packages;
- keep the Behat Composer environment resolvable for PHP 8.2;
- prefer Behat's built-in JUnit support when it satisfies the existing CI reporting contract.
Test design
Define and apply:
- a small semantic tag taxonomy;
- ownership boundaries between generic Nextcloud steps and LibreSign domain steps;
- modern Behat 4 context/feature conventions;
- independent scenario setup and lifecycle rules.
Tags describe test semantics. They must not be manually maintained shard identifiers.
Parallel execution
Use LibreCodeCoop/nextcloud-docker-development as the isolated Nextcloud environment for the complete parallel Behat database matrix. The worker lifecycle must support SQLite, MySQL, MariaDB 10.6/10.11 and PostgreSQL so the migration does not leave part of the compatibility matrix on a separate legacy execution architecture.
Each worker must have isolated mutable state, including where applicable:
- Nextcloud installation/configuration;
- database;
- data directory;
- users/groups;
- app configuration;
- generated files;
- mail state;
- mock/local services;
- temporary files;
- reports/logs.
Sharding must be deterministic and independent from semantic tagging.
Architecture
Shared package modernization
├── behat-builtin-extension
├── nextcloud-behat
│ ├── standalone assertions
│ ├── DataTable / DocString
│ └── reusable step modernization
├── mailpit-behat-extension
└── behat-tsa-extension
↓
LibreSign vendor-bin/behat + Behat 4
↓
LibreSign context/feature modernization
Test taxonomy ────────────────┐
Scenario concurrency safety ─┤
Deterministic sharding ──────┼─> isolated parallel Behat execution
Docker worker lifecycle ─────┘
↓
final Behat 3 cleanup + canonical documentation
Test-design principles
Scenarios must:
- be executable independently where intended for normal parallel execution;
- not rely on execution order;
- create or declare the state they require;
- not depend on state left by another scenario;
- avoid unnecessary implementation-detail steps;
- reuse generic Nextcloud steps when behavior is generic;
- keep LibreSign business-language steps in the LibreSign layer.
Parallel-execution principles
Parallelism must be introduced only after isolation prerequisites are proven.
Parallel workers belonging to one test run may share immutable caches/images, but must not share mutable application state.
Untrusted pull-request runs must not receive unnecessary repository write credentials or long-lived secrets, and mutable state must not be reused across unrelated test runs.
Performance
Record a serial baseline before selecting the final concurrency level.
Compare at least, where runner resources permit:
- one worker;
- two workers;
- four workers.
Measure:
- wall-clock duration;
- environment bootstrap duration;
- Behat execution duration;
- CPU/memory pressure where practical;
- total runner usage;
- failures/retries;
- evidence of flakiness or state leakage.
Do not select the final worker count without measurement.
Compatibility
- Target PHP >= 8.2.
- Target Behat ^4.0.
- New breaking releases of the shared testing packages do not retain Behat 3 compatibility.
- LibreSign changes target
main. - Do not backport this migration to stable branches.
Done when
- All required shared Behat packages have Composer-installable Behat 4-compatible breaking releases.
- LibreSign
mainexecutes its integration suite using Behat 4. - Behat tooling is isolated under
vendor-bin/behatand remains resolvable for PHP 8.2. - YAML Behat configuration is no longer required.
- Shared contexts use supported/public Behat APIs.
- PHPUnit is not used merely as the runtime assertion library of shared Behat contexts.
- Reusable step APIs use modern Behat argument types where applicable.
- Generic Nextcloud reusable steps and LibreSign domain steps have documented ownership boundaries and have been modernized accordingly.
- The test taxonomy is documented and applied.
- Scenarios intended for normal parallel execution are independent of execution order and shared mutable state.
-
nextcloud-docker-developmentsupports the complete Behat database matrix: SQLite, MySQL, MariaDB 10.6/10.11 and PostgreSQL. - Multiple isolated Nextcloud test environments can run concurrently.
- Deterministic sharding executes every intended test exactly once across workers.
- Parallel CI is measurably faster than the serial baseline.
- Parallel execution does not introduce measurable cross-worker state leakage or systematic flakiness.
- Failure output identifies the failing worker/shard and remains diagnosable.
- Existing database/Nextcloud/PHP compatibility coverage is preserved.
- Legacy Behat 3 infrastructure is removed after the new path is proven.
- The supported developer workflow is documented in LibreSign's canonical documentation.
Target
Branch: main
Milestone: Next Major (36)
Pull requests should implement focused child issues, not this epic.
- Ngôn ngữ chính
- PHP
- Star
- 844
- Fork
- 176
- Merge trung bình
- 14 giờ 3 phút
- Pull request đã merge (30 ngày)
- 344
Chuẩn bị môi trường
Khởi chạy dev container của dự án ngay trên trình duyệt, bằng tài khoản GitHub của bạn.
- Không có Dockerfile hay tệp Docker Compose
- Có mẫu pull request
- Đọc hướng dẫn đóng góp
Bắt đầu từ đâu
- Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
- Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
- Fork repository và làm thay đổi trên một nhánh.
- Mở pull request có tham chiếu số hiệu của issue.
Issue khác của LibreSign/libresign
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 78/100
Maintainer thường phản hồi trong vòng 1 ngày
-
OpenSSL engine: leaf certificates get O=Internet Widgits Pty Ltd, ST=Some-State from system openssl.cnfCó thể đã có người làm @vitormattos đã nhận 5 ngày trước. Đang mở
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 78/100
LibreSign/libresign#9366 · 1 người được giao ·
Maintainer thường phản hồi trong vòng 1 ngày
-
Độ khó 1/5 Dưới một giờ Mức phù hợp với người mới 83/100
LibreSign/libresign#9007 · 2 bình luận · 1 reaction ·
Maintainer thường phản hồi trong vòng 1 ngày
-
good first issue
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 78/100
LibreSign/libresign#8284 · 5 bình luận ·
Maintainer thường phản hồi trong vòng 1 ngày
-
frontend javascript technical debt
Độ khó 5/5 Hơn một tuần Mức phù hợp với người mới 8/100
Maintainer thường phản hồi trong vòng 1 ngày
Tất cả issue của LibreSign/libresign
Issue tương tự
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 79/100
Automattic/blocks-engine#6318 ·
Maintainer thường phản hồi trong vòng 1 ngày
-
[remove-resource] Delete Open Course Hub, Open Multicourse Hub, and Learn2 with GitSync skeletonsĐang mở
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 68/100
-
Độ khó 1/5 Dưới một giờ Mức phù hợp với người mới 88/100
-
Allow @ in the service nameĐang mở
Độ khó 1/5 Dưới một giờ Mức phù hợp với người mới 82/100
-
35-feedback
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 78/100
Maintainer thường phản hồi trong vòng 1 ngày