General testing strategy of crates in CI

未關閉
#28 1 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視

還沒有人認領這個 Issue。

評估

難度
5/5
預估耗時
一週以上
新手友好度
25/100
Issue 類型
功能
描述清晰度
需要釐清
活躍度
停滯
技術堆疊
github-actions, rust
領域
ci-cd, testing

研究方向

Start by reviewing the workspace Cargo.lock, Cargo.toml dependency patches, and existing CI jobs for stable, MSRV, and minimal-versions testing. The issue proposes three dependency resolution scenarios but leaves key choices unresolved, including lockfile policy and how to handle patches. A concrete outcome would be an agreed testing strategy and CI changes covering the chosen scenarios.

由索引模型根據 Issue 內容生成。

描述

Most of our repositories contain multiple crates which are part of the same workspace. We also have a number of inter-repository dependencies (e.g. digest depends on crypto-common). In such cases instead of using path while specifying dependency in the crate's Cargo.toml we use dependency patching (e.g. see here).

Currently we commit workspace's Cargo.lock and update it periodically using Dependabot. It allows us to isolate CI failures caused by upstream dependencies. It also reduces CI times a bit since we don't need to generate lock files from scratch. We use the same Cargo.lock for both MSRV and stable tests. MSRV tests use versions manually provided in the CI config.

Additionally, we have minimal-versions jobs which test crates separately from the repository workspace using unstable -Z minimal-versions option. Unfortunately, testing it outside of workspace means that dependency patching defined on the workspace level does not works and we have to manually disable such jobs while patching is present.

In the new releases we plan to rely on the MSRV-aware resolver meaning that we can update MSRV in patch releases, which can lead to mixed-MSRV workspaces. Considering how the MSRV-aware resolver works by default (it selects dependency versions compatible with the declared MSRV), it may be problematic.

Imagine a repo with crates A and B with MSRV equal to 1.85, now in a new version of A we want to use dependency C with MSRV 1.90. We bump MSRV of crate A accordingly, but now we have a problem: the resolver is unable to generate Cargo.lock compatible with MSRV of crate B.

Going forward, I think we should test at least 3 scenarios for all crates:

  1. Stable toolchain with latest semver-compatible dependencies (i.e. generated with cargo update --ignore-rust-version)
  2. MSRV toolchain acquired from the rust-version field with latest MSRV-compatible dependencies
  3. Nightly toolchain with minimal semver-compatible version (i.e. generated with -Z minimal-versions)

This list would require the following changes:

  • If we want to continue committing Cargo.lock we should add .cargo/config.toml to every repository with incompatible-rust-versions = "allow" to satisfy 1.
  • To satisfy 2 we would need to add extraction of the rust-version value (see this URLO thread) and to exclude the crate from the repo workspace.

Unresolved questions:

  • Should we continue to commit Cargo.lock? I don't remember many cases of upstream dependencies breaking CI, so it may not be important. We could replace it with a cron job which would test freshest upstream dependencies periodically.
  • Should we test on Nightly and/or Beta with latest semver-compatible dependencies?
  • Should we use MSRV toolchain to run tests minimal-versions jobs (i.e. use Nightly to generate lock file and switch to MSRV toolchain for cargo test)?
  • How to deal with patch issues in 2 and 3?
主要語言
沒有語言資料
星號
13
分支
4
平均合併
41 分鐘
30 天內合併 PR
1

環境準備

這個專案沒有提供開發容器、Dockerfile 或貢獻指南,環境需要你自己搭建:先看它的 README,通用步驟見我們的新手貢獻指南。

從這裡開始

  1. 先讀完整個 Issue,再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫,在一個分支上完成修改。
  4. 送出 Pull Request,並在描述裡引用這個 Issue 編號。

RustCrypto/meta 的其他 Issue

查看 RustCrypto/meta 的全部 Issue

相似的 Issue

更多 DevOps Issue

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。