[Code scan] NEB zero failure rates are emitted as missing metrics
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 88/100
Research direction
Start in lambench/tasks/calculator/neb/neb.py at lines 62-86 and trace how NEB failure counts and denominators are grouped. Add the regression case described in the issue for a type with no failures, then verify the resulting metric is emitted as 0.0 rather than missing.
Written by the indexing model from the issue text.
Description
This issue was found by a Codex global repository scan of tracked non-test files at commit 8c93925cb10b401b2b83c738bd9263fd74474468.
Relevant code
Impact
The NEB failure percentage is computed from only the rows where error > ERROR_THRESHOLD. If a reaction type has zero failures, that type is absent from the numerator groupby, so the division can produce a missing value instead of an explicit 0.0 failure rate.
A perfect category should be represented as 0.0, not as a missing metric. Otherwise downstream JSON and leaderboard plots can treat a valid result as absent.
Suggested fix
Build the denominator counts first, reindex the failure counts against those types with fill_value=0, then divide. Add a regression case where one NEB type has no failures.
- Dominant language
- Python
- Stars
- 20
- Forks
- 11
- Avg merge
- 30m
- Merged PRs (30d)
- 4
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from deepmodeling/LAMBench
-
Difficulty 1/5 Under an hour Newbie friendliness 86/100
deepmodeling/LAMBench#443 ·
-
[Code scan] Missing NVE systems crash stability scoring instead of receiving the failure penalty Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
deepmodeling/LAMBench#440 ·
-
Difficulty 2/5 Half a day Newbie friendliness 74/100
deepmodeling/LAMBench#439 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
deepmodeling/LAMBench#438 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
deepmodeling/LAMBench#436 ·
All issues in deepmodeling/LAMBench
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100