Adjust free space threshold for jenkins and allow it to bring machines offline
维护者通常 3 天内回复
还没有人认领这个 Issue。
评估
- 难度
- 3/5
- 预计耗时
- 1-2 天
- 新手友好度
- 48/100
- Issue 类型
- 功能
- 描述清晰度
- 基本清楚
- 活跃度
- 活跃
- 技术栈
- yaml
- 领域
- ci-cd, infrastructure
调研方向
Start by locating the Jenkins configuration that sets the workspace free-space warning threshold and controls whether low-space agents are taken offline; the issue does not name the file. Review how the current 0.2GB setting is applied, then determine how to set the threshold to roughly 4–5GB and enable taking agents offline. Done when the policy prevents builds from being scheduled on agents below the chosen threshold.
由索引模型根据 Issue 内容生成。
描述
Jenkins has the option to mark machines offline if the workspace space gets too low. At the moment we hae the warning threshold set to 0.2G of space (i.e. when it gets below that it will warn, but not take the machine offline).
I think we should be more aggressive with this and set it to a larger figure (At least 2GB but ideally 4-5GB which would usually cover one full build on most systems) and allow jenkins to take machines offline when they hit the threshold which should prevent out of space conditions.
At the time of writing having a policy of "offline at 5GB" would take out the following executors, including the IBM Cloud RHEL ones which we raised a separate issue for.
Since I have the page up on my screen just now I'll add a note that the following systems have between 5GB and 10GB free:
In recent months we have had quite a lot of issues relating to systems going low on disk space and while taking the machines offline prevents things being scheduled on them and therefore potentially causing queue backlogs (which is why we usually have redundancy across systems for each build type) that is probably preferable to having CI builds fail so I propropose we allow jenkins to take machines in that state machines offline.
Richard has looked at single points of failures in our critical servers under https://github.com/nodejs/build/issues/4498 but we should perhaps take a closer look at our regular agents, not least because we hit a situation today where the only two agents that could run one particuar build were on separate containers on a single physical host and therefore were impacted simultaneously: https://github.com/nodejs/build/issues/4534
- 主要语言
- Jinja
- 星标
- 544
- 派生
- 184
- 平均合并
- 4 天 16 小时
- 30 天内合并 PR
- 12
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
nodejs/build 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 65/100
nodejs/build#4539 · 1 个 reaction ·
维护者通常 3 天内回复
-
build-agenda
难度 2/5 1-3 小时 新手友好度 65/100
维护者通常 3 天内回复
-
难度 2/5 1-3 小时 新手友好度 72/100
维护者通常 3 天内回复
-
build-agenda
难度 4/5 3-5 天 新手友好度 48/100
维护者通常 3 天内回复
-
incident
难度 4/5 3-5 天 新手友好度 38/100
维护者通常 3 天内回复
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 67/100
MISP/misp-workbench#496 ·
维护者通常 1 天内回复
-
bug
难度 2/5 1-3 小时 新手友好度 84/100
praetorianer777/Dynamo#53 ·
维护者通常 1 天内回复
-
internal test setup
难度 2/5 1-3 小时 新手友好度 76/100
SAP/terraform-provider-scc#430 ·
-
internal test setup
难度 2/5 1-3 小时 新手友好度 78/100
SAP/terraform-provider-btp#1743 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 72/100
MISP/misp-docker#478 ·
维护者通常 1 天内回复