test_effect_timing's 120 s subprocess timeout is still wall-clock and fails under machine load
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 69/100
Research direction
Start with tests/test_timings.py::test_effect_timing and inspect how its subprocess timeout relates to the CPU-time throughput assertion added in #688. Keep CPU time as the performance check and make the wall-clock limit a hang guard; the issue suggests a generous timeout or a child CPU limit via resource.setrlimit on POSIX. Done when the test remains protected from hangs without failing solely because the machine is under load; consider the regression test or documentation mentioned in the issue.
Written by the indexing model from the issue text.
Description
tests/test_timings.py::test_effect_timing runs the throughput check in a fresh interpreter with subprocess.run(..., timeout=120). #688 moved the throughput budget to CPU time because wall-clock time counts every other process on the machine (#679). The outer 120 s limit is still wall-clock, so the same problem reappears one layer out.
Seen 2026-11-05 during ./deploy.sh 11.6.1 on a shared dev machine with a load average of about 166. Other suites, including another deploy, were running at the same time. The subprocess's imports, reference loading and warmup exceeded 120 s of wall time:
FAILED tests/test_timings.py::test_effect_timing - subprocess.TimeoutExpired:...
================= 1 failed, 4507 passed in 2223.04s (0:37:03) ==================
deploy.sh then stopped before uploading, as it should. A retry under normal load is expected to pass, and CI runners don't hit this.
Suggested fix: keep the CPU-time budget as the real assertion, and make the subprocess timeout only a hang guard, generous enough to be load-insensitive (for example several minutes, or scaled by os.getloadavg() against the CPU count). Alternatively, apply the limit to the child's CPU time via resource.setrlimit(RLIMIT_CPU) in preexec_fn on POSIX. Add a regression test that the guard is not wall-clock-only, or document why it must be.
- Dominant language
- Python
- Stars
- 88
- Forks
- 28
- Avg merge
- 4h 34m
- Merged PRs (30d)
- 52
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from openvax/varcode
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
Maintainers usually reply within 1 day
-
MAF reader: 'Too few columns' error does not name the missing columnsPossibly taken A pull request linked to this issue is open or already merged. Open
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
load_maf does not convert UCSC contig names (GDC chr-prefixed MAFs fail annotation)Possibly taken A pull request linked to this issue is open or already merged. Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 45/100
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
Maintainers usually reply within 1 day
Similar issues
-
binding:python bug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
The Browser tab fails with a Wayland error when Open Terminal runs from a Wayland desktop sessionOpen
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
open-webui/open-terminal#210 ·
-
triage/confirmed
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
agentscope-ai/agentscope#3807 ·
Maintainers usually reply within 2 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
NousResearch/hermes-agent#150542 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 91/100
triton-inference-server/server#9069 ·
Maintainers usually reply within 1 day