Can resumed CI reuse successful matrix configurations?
まだ誰も着手していません。
評価
- 難易度
- 5/5
- 見積もり時間
- 1週間以上
- 初心者へのやさしさ
- 42/100
- issue の種類
- 機能追加
- 明瞭さ
- おおむね明確
- 活発さ
- 活発
- 技術スタック
- groovy
- 領域
- build-system, ci-cd
調査の方向性
jenkins/scripts/VersionSelectorScript.groovy と node-test-commit-linux matrix-selection script から始め、次に MultiJobResumeControl と Jenkins のマトリックス再開動作を追跡します。提案されたオプトインを、文書化された成功、失敗、再度の再開、壊れたメタデータ、すべてスキップされた場合のケースに対して検証します。完了の条件は、オプトインしていないジョブを変更せずに、成功した構成が再利用されることです。
索引モデルが issue の本文から書いたものです。
説明
Refs: https://openjs-foundation.slack.com/archives/C03BJP63CH0/p1787917001257859
I do not know Jenkins well enough to be confident about the implementation details. I asked an LLM to inspect the relevant build metadata, job configuration, and plugin sources. This is how I understand its suggestion.
Resume already seems to work between Multijob children. For example, node-test-commit #91372 reused the same builds for the twelve successful direct children from #91368.
The gap seems to be inside matrix jobs.
This uses node-test-commit-linux as a worked example. The intended scope is eventually all matrix jobs under node-test-commit, after checking their configurations individually.
node-test-commit-linux #72543 had ten configurations. Eight succeeded, alpine-last-latest-x64 was UNSTABLE, and rhel8-x64 was FAILURE. Its resume, #72548, ran all ten configurations again.
AFAICT, Multijob sees node-test-commit-linux as one unsuccessful child. It starts a new matrix parent, whose selector then selects every eligible configuration without considering the individual results from the build being resumed.
The suggested change is to extend VersionSelectorScript.groovy. Multijob already passes the previous child build through MultiJobResumeControl, so the selector could follow that lineage and exclude configurations whose newest exact result was SUCCESS.
The selector is also used by jobs outside the normal node-test-commit hierarchy. The proposed behavior is therefore opt-in and defaults to off. The example enables it for node-test-commit-linux first; the other matrix jobs under node-test-commit could opt in after being checked.
Suggested VersionSelectorScript.groovy diff
diff --git a/jenkins/scripts/VersionSelectorScript.groovy b/jenkins/scripts/VersionSelectorScript.groovy
--- a/jenkins/scripts/VersionSelectorScript.groovy
+++ b/jenkins/scripts/VersionSelectorScript.groovy
@@ -97,26 +97,109 @@
// NOTE: this assumes that the default "Agents"->"Name" in the Configuration
// Matrix is left as "nodes", if it's changed then `it.nodes` below won't work
// and returning a result with a "nodes" property won't work.
result['nodes'] = []
// Before running this script, `def buildType = 'release'` or some other value
// to be able to use the appropriate `buildType` in the exclusions
def _buildType
try {
_buildType = buildType
} catch (groovy.lang.MissingPropertyException e) {
_buildType = 'test'
}
combinations.each{
def builderLabel = it.nodes
// Default to running all builders if nodeMajorVersion is still -1
// (i.e. the version check failed)
if (nodeMajorVersion >= 4) {
if (!canBuild(nodeMajorVersion, builderLabel, _buildType)) {
println "Skipping $builderLabel for Node.js $nodeMajorVersion"
return
}
}
result['nodes'].add(it)
}
+
+// This is opt-in because this script is shared by jobs outside
+// node-test-commit.
+if (!(binding.hasVariable('reuseSuccessfulConfigurations') &&
+ binding.getVariable('reuseSuccessfulConfigurations') == true))
+ return
+
+// A resumed Multijob child carries the MatrixBuild it is resuming.
+// Avoid importing the action class because this script is evaluated by a
+// plugin whose class loader does not declare a Multijob dependency.
+def resumeControlClassName =
+ 'com.tikal.jenkins.plugins.multijob.MultiJobResumeControl'
+def currentBuild = execution.build
+
+try {
+ def resumeControl = currentBuild.actions.find {
+ it.class.name == resumeControlClassName
+ }
+ def resumeSource = resumeControl?.run
+ def previousBuild = resumeSource
+ def latestResults = [:]
+ def seenBuilds = [] as Set
+
+ // Follow the complete resume lineage. getRuns() is deliberately not used:
+ // it may inherit a run from an unrelated matrix build which ran in between.
+ while (previousBuild != null) {
+ if (previousBuild.class.name != 'hudson.matrix.MatrixBuild' ||
+ previousBuild.parent != currentBuild.parent) {
+ throw new IllegalStateException(
+ "Unexpected resumed build ${previousBuild.externalizableId}")
+ }
+
+ if (!seenBuilds.add(previousBuild.externalizableId)) {
+ throw new IllegalStateException(
+ "Cycle in resume lineage at ${previousBuild.externalizableId}")
+ }
+
+ // The first exact run found for a combination is its newest result.
+ previousBuild.exactRuns.each { run ->
+ def combination = run.parent.combination
+ if (!latestResults.containsKey(combination))
+ latestResults[combination] = run.result
+ }
+
+ resumeControl = previousBuild.actions.find {
+ it.class.name == resumeControlClassName
+ }
+ previousBuild = resumeControl?.run
+ }
+
+ def successfulResumeCombinations = [] as Set
+ latestResults.each { combination, previousResult ->
+ if (previousResult?.toString() == 'SUCCESS')
+ successfulResumeCombinations.add(combination)
+ }
+
+ // Apply this after the normal version and MACHINES selection.
+ // If it would select no work, rerun the eligible set so a failure outside
+ // the individual cells cannot become a no-op success.
+ def eligibleCombinations = result['nodes']
+ def combinationsToRun = eligibleCombinations.findAll {
+ !successfulResumeCombinations.contains(it)
+ }
+ int skippedCount =
+ eligibleCombinations.size() - combinationsToRun.size()
+
+ if (skippedCount > 0) {
+ if (combinationsToRun.isEmpty()) {
+ println 'Resume would skip every eligible combination; running all'
+ } else {
+ println "Resume: reusing ${skippedCount} successful combination(s)"
+ // Otherwise Jenkins defaults to the immediately preceding matrix build,
+ // which may be unrelated to this resume.
+ currentBuild.setBaseBuild(resumeSource)
+ result['nodes'] = combinationsToRun
+ }
+ }
+} catch (Exception e) {
+ // Keep the normal selection on malformed or unexpected resume metadata.
+ println "Resume metadata could not be read; running all eligible " +
+ "combinations (${e.class.name}: ${e.message})"
+}
The node-test-commit-linux matrix-selection script would initially opt in with:
buildType = 'test'
+reuseSuccessfulConfigurations = true
evaluate(code)
My understanding of the proposed behavior is:
- the existing version and
MACHINESselection still runs first - jobs which do not opt in retain their current behavior
- only exact
SUCCESSresults are reused UNSTABLE,FAILURE,ABORTED,NOT_BUILT, and missing configurations run again- repeated resumes follow the explicit resume lineage
- malformed or unexpected resume metadata leaves the existing full selection unchanged
- if every configuration would be skipped, the eligible set runs again in case the matrix parent failed outside an individual configuration
The setBaseBuild() call is apparently needed so Jenkins inherits results from the matrix build actually being resumed, rather than whichever unrelated matrix build happened to run immediately before it.
Does this match how the matrix jobs are intended to resume? In particular:
- Is
VersionSelectorScript.groovythe right place to carry the existing Multijob resume behavior into matrix configurations? - Is making the behavior opt-in appropriate given the selector's other consumers?
- If it works as expected for Linux, should the other matrix jobs under
node-test-commitopt in as well?
- 主要言語
- Jinja
- スター
- 541
- フォーク
- 185
- 平均マージ
- 2日 18時間
- マージ済み PR(30日)
- 6
コントリビューションガイド
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
nodejs/build のほかの issue
-
難易度 2/5 1〜3時間 初心者へのやさしさ 70/100
-
platform:ppc
難易度 1/5 1時間未満 初心者へのやさしさ 65/100
-
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
-
incident platform:arm
難易度 4/5 3〜5日 初心者へのやさしさ 35/100
-
難易度 4/5 3〜5日 初心者へのやさしさ 45/100
似ている issue
-
core dependencies
難易度 1/5 1時間未満 初心者へのやさしさ 80/100
-
bug github_actions
難易度 2/5 1〜3時間 初心者へのやさしさ 75/100
registrystack/registry-stack#1393 ·
-
難易度 2/5 1〜3時間 初心者へのやさしさ 70/100
-
Name consistency オープン
難易度 2/5 1〜3時間 初心者へのやさしさ 75/100
eellak/triplestore#65 · コメント 1 件 ·
-
難易度 2/5 1〜3時間 初心者へのやさしさ 65/100