Job scheduling efficiency improvements
还没有人认领这个 Issue。
评估
调研方向
未指定任何文件、测试或入口点。首先定位 OpenTitan 夜间和每周回归所使用的 scheduler，然后将未勾选的目标拆分为范围彼此独立的工作。只有就计划持久化、考虑时间或资源的限制，或考虑许可证的调度达成一致的范围和验收标准，才能算作完成。
由索引模型根据 Issue 内容生成。
描述
The nightly and weekly regressions for OpenTitan take significant resources to run. It is helpful to have more flexibility in the scheduler so that we can make better use of those resources.
Some ideas:
- Test ordering - for the nightly regression runs most of the tests are complete in the first hour or so. By starting the longest running tests first and infilling with shorter running tests we can make the overall regression run time shorter, but keeping our scheduling slots busy.
- Split run schedule
- Generate a schedule and save to disk
- Restore the schedule and run it
- Restore the schedule and run a subset - potentially on a different machine
- Restore a schedule and based on the working files resume a WIP run
- More refined max parallel job control
- Tool specific max parallel - different tools have different resource requirements so it's appropriate to have different limits proportional to the resource usage
- Time based job control - it's common to run tests overnight/weekends when there may less contention of resources (compute or licenses). However jobs may have to be stopped if they don't finish within the expected timeslot if they are starving resources from other users. It would be useful to have the ability to set time dependent resource usage limits, so big jobs can be started any time and make use of the available resources at the time.
- License aware scheduling - periodically query the license server so that the number of parallel jobs can be dynamically changed to reflect the currently available licenses. Avoiding starving other users, while making more efficient use of licenses.
- General resource aware scheduling - periodically poll an external service to know how much resource is required for each job. This mechanism could be used for license server query, but also potentially other things like the machines memory and CPU usage.
- 主要语言
- Python
- 星标
- 16
- 派生
- 22
- 平均合并
- 1 天 11 小时
- 30 天内合并 PR
- 5
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
lowRISC/dvsim 的其他 Issue
-
bug
难度 2/5 1-3 小时 新手友好度 84/100
-
难度 2/5 1-3 小时 新手友好度 68/100
-
难度 2/5 1-3 小时 新手友好度 72/100
-
难度 4/5 3-5 天 新手友好度 35/100
-
Better log visibility on the license type usage profile可能已有人在做 @AlexJones0 于 22 天前认领。 未关闭
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 77/100
developmentseed/titiler-cmr#250 · 1 条评论 ·
维护者通常 1 天内回复
-
[Bug]: IndexHandler.post crashes with HTTP 500 on malformed payloads, missing credentials, or when login is disabled可能已有人在做 关联的 PR 仍在进行中或已合并。 未关闭
难度 2/5 1-3 小时 新手友好度 78/100
维护者通常 6 天内回复
-
bug
难度 2/5 1-3 小时 新手友好度 76/100
cookiecutter/cookiecutter-django#6999 ·
维护者通常 1 天内回复
-
fix: @_helpers.positional raises TypeError when decorating functions without default arguments可能已有人在做 @codeXsidd 今天认领。 未关闭
难度 1/5 1 小时以内 新手友好度 88/100
googleapis/google-api-python-client#2845 ·
维护者通常 1 天内回复
-
github_actions
难度 2/5 1-3 小时 新手友好度 65/100
collective/icalendar#2003 · 1 条评论 ·
维护者通常 1 天内回复