Solution runner can exceed the configured timeout and keep CI running
維護者通常 1 天內回覆
評估
- 難度
- 4/5
- 預估耗時
- 3-5 天
- 新手友好度
- 48/100
研究方向
Start by reading run.py and the GitHub Actions workflow; trace how the per-test timeout is applied and how compilation is launched. Run the existing test suite and inspect how the runner handles timed-out processes. Done means compilation and total execution have separate limits, spawned processes are cleaned up, and the workflow has an appropriate job timeout.
由索引模型根據 Issue 內容生成。
描述
I noticed that the solution runner doesn't properly limit how long a submitted solution can run.
The configured timeout is applied separately to each test case rather than to the solution's total execution time. Compilation also doesn't appear to have any timeout.
For example, I tested four cases that each took around 0.35 seconds. All four passed with a configured timeout of just 1 second, even though the total execution took approximately 1.55 seconds.
With the current behavior, 100 test cases and a 10-second timeout could potentially keep a solution running for around 1,000 seconds. A compilation process that hangs could also block the runner indefinitely.
I expected the runner to enforce an overall execution limit so that a submitted solution couldn't occupy CI resources for much longer than intended.
Looking through run.py, the timeout is applied inside the loop that executes individual test cases, while the compilation step has no timeout. The GitHub Actions workflow also doesn't define a job-level timeout.
This could become a bigger problem with submissions that run slowly, generate excessive output, or start additional processes.
It would be useful to enforce separate compilation and overall execution limits, along with proper cleanup of spawned processes and safeguards against excessive resource usage.
- 主要語言
- Python
- 星號
- 1
- 分支
- 6
- 平均合併
- 23 小時 30 分鐘
- 30 天內合併 PR
- 20
環境準備
- 沒有 Dockerfile 或 Docker Compose 檔案
- 有 Pull Request 範本
- 沒有貢獻指南
從這裡開始
- 先讀完整個 Issue，再讀專案的貢獻指南。
- 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
- Fork 儲存庫，在一個分支上完成修改。
- 送出 Pull Request，並在描述裡引用這個 Issue 編號。
techcsispit/code-corner 的其他 Issue
-
prime/solution.java doesn't compile: public class PrimeCheck in solution.java, README requires Solution.java with a Solution class可能已有人在做 @tanujb03 於 5 天前認領。 未關閉bug good first issue
難度 1/5 1 小時以內 新手友好度 82/100
techcsispit/code-corner#35 ·
維護者通常 1 天內回覆
-
Failed compilations always report 0.000 seconds可能已有人在做 @Nik-2208 於 3 天前認領。 未關閉bug good first issue
難度 2/5 1-3 小時 新手友好度 35/100
techcsispit/code-corner#37 · 1 則留言 · 已指派 1 人 ·
維護者通常 1 天內回覆
-
Add test cases to a problem可能已有人在做 @25crns 於 12 天前認領。 未關閉good first issue tests
techcsispit/code-corner#2 · 3 則留言 · 已指派 1 人 ·
維護者通常 1 天內回覆
查看 techcsispit/code-corner 的全部 Issue
相似的 Issue
-
難度 2/5 1-3 小時 新手友好度 72/100
huggingface/sentence-transformers#4186 ·
維護者通常 5 天內回覆
-
documentation good first issue
難度 2/5 半天 新手友好度 72/100
sujithputta02/Esapay#13 ·
-
bot-triaged module: correctness (silent) module: mps module: NaNs and Infs triage review
難度 2/5 半天 新手友好度 62/100
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 68/100
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 62/100
SCIInstitute/SCIRun#2842 ·
維護者通常 3 天內回覆