ci: speed up PR feedback (remove -p 1, shard e2e, cache, drop multi-arch on PR)
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 58/100
- Issue 类型
- 重构
- 描述清晰度
- 描述清楚
- 活跃度
- 冷清
- 领域
- build-system, ci-cd, devops, performance
调研方向
从 .github/workflows/go-test.yaml、.github/workflows/e2e-tests.yaml、.github/workflows/build-images.yaml、Dockerfile 和 e2e-tests/run-tests.sh 开始。在更改独立的 CI 步骤之前,验证当前 workflow 的耗时,并运行受影响的 Go 和端到端检查。完成的标准是:并行 Go 测试、预热的 Docker 缓存、仅针对 amd64 的 PR 镜像构建以及基于 suite 的 e2e 作业都成功完成,且不更改测试代码。
由索引模型根据 Issue 内容生成。
描述
Context
PR wall-clock to merge readiness is ~13 min, dominated by the Go test workflow (12m15s test step) and E2E tests (~10m45s). Audit of the workflows + a local empirical pass shows the bulk is achievable without test-code refactor and without sharding. This issue lists the prioritized findings as discrete deliverables.
Related: PR #4250 (CircleCI → GHA migration), commit 05a70569.
Findings
1. Drop -p 1 from go-test.yaml (highest impact, lowest effort)
.github/workflows/go-test.yaml:28 runs go test -p 1 ./... -race -coverprofile=c_raw.out. The -p 1 was inherited verbatim from .circleci/config.yml where it sat inside circleci tests split (8-way shard). The GHA migration dropped the shard but kept the flag — its original purpose (per-shard serialization) no longer applies.
Audit for inter-package parallelism hazards on the current codebase:
| Hazard | Found |
|---|---|
net.Listen to fixed port |
0 (2 files call net.Listen: storage/session_memcached_test.go:139 uses :0, network/transport/grpc/connection_manager_test.go:63 uses bufconn) |
os.Chdir in tests |
0 |
TestMain setting up shared infra |
0 |
| Shared filesystem paths | 0 (the ./test/* reads in auth/services/selfsigned/ are package-local; Go sets cwd per package) |
os.MkdirTemp to fixed path |
0 |
go test -p N runs N test binaries in parallel, each in its own process — so the 49 os.Setenv/t.Setenv call sites are isolated by process boundary, not a hazard either.
Empirical local run on 12-core machine:
| Config | Wall-clock | Outcome |
|---|---|---|
go test -p 1 -race ./... |
7m 04s | PASS |
go test -p 2 -race ./... |
3m 31s | PASS |
go test -race ./... (default) |
2m 06s | PASS |
GHA ubuntu-latest is 2 vCPUs, so dropping -p 1 means the default GOMAXPROCS=2 parallelism kicks in. Expected CI saving: ~6 min/PR (12m15s → ~6m).
Change: drop -p 1. One line.
2. Add Docker layer caching for the e2e build
.github/workflows/e2e-tests.yaml:55, 67 — both Build and push steps lack cache-from/cache-to. Every PR rebuilds the Go binary from scratch (~1m 46s).
- name: Build and push
uses: docker/build-push-action@v7
with:
cache-from: type=gha,scope=e2e
cache-to: type=gha,scope=e2e,mode=max
# ... existing fields
Dockerfile layer order at Dockerfile:16-21 already puts go.mod/go.sum + go mod download before COPY . ., so the cache is effective on PRs that don't touch go.sum. ~1–2 min/PR after warm-up.
3. Drop multi-arch on PR runs in build-images.yaml
.github/workflows/build-images.yaml:80 builds linux/amd64,linux/arm64 on every PR. arm64 under QEMU is ~5–8× slower than native — it's the bulk of the 26 min master run. On PRs push: false (line 81), so we only verify the build doesn't break; arm64 is redundant.
platforms: ${{ github.event_name == 'pull_request' && 'linux/amd64' || 'linux/amd64,linux/arm64' }}
Workflow isn't a required check, but it consumes ~20 runner-min/PR. On master/tags, separately, consider switching arm64 to native ubuntu-24.04-arm runners (free for public repos) — cuts the 26 min master build to ~5 min as a parallel two-job matrix.
4. Shard e2e-tests by suite (bigger refactor)
e2e-tests/run-tests.sh runs 12+ docker-compose suites sequentially (~8m 25s of the 10m 45s job). Suites are independent (separate docker-compose stacks, no shared state).
Change:
- Refactor
run-tests.shto accept a suite name (./run-tests.sh oauth-flow). - Split
.github/workflows/e2e-tests.yamlinto abuildjob (producesnuts-node-ci:$SHA, pushes to GHCR once) →test-<suite>matrix jobs (each pulls the image and runs one suite).
4-shard split → ~5 min/PR saving (longest suite on critical path).
Suggested actions
| # | Description | Effort | Independent? |
|---|---|---|---|
| 1 | Remove -p 1 from go-test.yaml |
S | yes |
| 2 | Add gha cache to e2e Docker build | S | yes |
| 3 | Conditional platforms on build-images.yaml |
S | yes |
| 4 | Shard e2e-tests by suite |
M | yes |
(1), (2), (3) can land as three small PRs first; (4) is the bigger structural change.
Out of scope
- Fix hardcoded port strings in tests (used in config-validation, not actual binds — see audit table above for why they're not a parallelism hazard). Refactor would let us go beyond 2-core parallelism without sharding, but isn't needed to reach the ~4–5 min/PR target.
- Larger runners (
ubuntu-latest-4-cores). Linear speedup at 2× per-minute cost — net-neutral on spend, faster wall-clock. Not necessary if (1)+(4) bring PR wall-clock to ~4–5 min. - Skip CodeQL on dependency-only PRs. Not in required checks; out of scope here.
Related
- Triggering PR: #4250 (CircleCI → GHA migration that introduced the regression)
- Commit that introduced
-p 1in the new workflow: 05a70569
- 主要语言
- Go
- 星标
- 29
- 派生
- 24
- 平均合并
- 2 天 5 小时
- 30 天内合并 PR
- 37
环境准备
- 提供 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 没有贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
nuts-foundation/nuts-node 的其他 Issue
-
bug
难度 2/5 1-3 小时 新手友好度 72/100
nuts-foundation/nuts-node#4450 ·
维护者通常 1 天内回复
-
bug discuss
难度 1/5 1 小时以内 新手友好度 88/100
nuts-foundation/nuts-node#4436 ·
维护者通常 1 天内回复
-
auth bug
难度 2/5 1-3 小时 新手友好度 82/100
nuts-foundation/nuts-node#4435 ·
维护者通常 1 天内回复
-
bug
难度 2/5 1-3 小时 新手友好度 72/100
nuts-foundation/nuts-node#4321 ·
维护者通常 1 天内回复
-
难度 1/5 1 小时以内 新手友好度 88/100
nuts-foundation/nuts-node#4230 ·
维护者通常 1 天内回复
查看 nuts-foundation/nuts-node 的全部 Issue
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 78/100
FootprintAI/Containarium#3182 ·
维护者通常 1 天内回复
-
bug from-studio
难度 2/5 1-3 小时 新手友好度 62/100
esengine/DeepSeek-Reasonix#14608 ·
维护者通常 1 天内回复
-
triage
难度 2/5 1-3 小时 新手友好度 75/100
getarcaneapp/arcane#4590 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 78/100
维护者通常 1 天内回复
-
Library panel usage dialog uses singular grammar可能已有人在做 关联的 PR 仍在进行中或已合并。 未关闭area/frontend/library-panels automated-triage type/bug type/ux
难度 1/5 1 小时以内 新手友好度 78/100
维护者通常 1 天内回复