[Code scan] Pressure task reports success_rate with the wrong denominator
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 74/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- python
- Domain
- performance
Research direction
Start in lambench/tasks/calculator/pressure/pressure.py at lines 57-84 and trace how successful samples, failed attempts, and the final metric are counted. Exercise the pressure task with mixed and all-failing samples, then update the calculation so success uses total attempts and no-success cases return controlled metrics with a 0.0 success rate.
Written by the indexing model from the issue text.
Description
This issue was found by a Codex global repository scan of tracked non-test files at commit 8c93925cb10b401b2b83c738bd9263fd74474468.
Relevant code
Impact
num_samples is incremented only after a successful calculation, while num_fails is incremented for failed attempts. The final formula is (num_samples - num_fails) / num_samples, which can underreport success and can crash when every sample fails.
For example, with one successful sample and one failed sample, the current formula reports (1 - 1) / 1 = 0.0 instead of the expected 1 / 2 = 0.5.
Suggested fix
Track total attempts separately from successful samples. Compute success_rate = num_successes / total_attempts, and return controlled NaN or None metrics plus 0.0 success when there are no successful samples.
- Dominant language
- Python
- Stars
- 20
- Forks
- 11
- Avg merge
- 30m
- Merged PRs (30d)
- 4
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from deepmodeling/LAMBench
-
Difficulty 1/5 Under an hour Newbie friendliness 86/100
deepmodeling/LAMBench#443 ·
-
[Code scan] Missing NVE systems crash stability scoring instead of receiving the failure penalty Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
deepmodeling/LAMBench#440 ·
-
Difficulty 2/5 Half a day Newbie friendliness 74/100
deepmodeling/LAMBench#439 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
deepmodeling/LAMBench#438 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
deepmodeling/LAMBench#436 ·
All issues in deepmodeling/LAMBench
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100