[Code scan] Missing or failed inference-efficiency results can crash or change schema
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- Half a day
- Newbie friendliness
- 74/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Quiet
- Tech stack
- python
- Domain
- performance
Research direction
Start in lambench/metrics/vishelper/results_fetcher.py around lines 108-119 and trace the result handling into lambench/metrics/utils.py around lines 129-150. Check the zero-, multiple-, success-, and failure-result paths. Done means missing or ambiguous records no longer crash, and generated JSON uses the same result keys in both success and failure cases.
Written by the indexing model from the issue text.
Description
This issue was found by a Codex global repository scan of tracked non-test files at commit 8c93925cb10b401b2b83c738bd9263fd74474468.
Relevant code
https://github.com/deepmodeling/LAMBench/blob/8c93925cb10b401b2b83c738bd9263fd74474468/lambench/metrics/vishelper/results_fetcher.py#L108-L119
https://github.com/deepmodeling/LAMBench/blob/8c93925cb10b401b2b83c738bd9263fd74474468/lambench/metrics/utils.py#L129-L150
Impact
fetch_inference_efficiency_results_for_one_model() logs a warning when the database query returns zero or multiple records, but it still indexes task_results[0]. A missing record therefore raises IndexError during result generation.
There is also a schema inconsistency: the successful aggregation branch returns standard_deviation, while the failure branch returns std_time. Consumers of the generated JSON see different keys depending on whether any system failed.
Suggested fix
Return None or a consistent failure object when len(task_results) != 1, before indexing. Also make the failure branch return the same keys as the success branch, for example average_time, standard_deviation, and success_rate.
- Dominant language
- Python
- Stars
- 20
- Forks
- 11
- Avg merge
- 30m
- Merged PRs (30d)
- 4
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from deepmodeling/LAMBench
-
Difficulty 1/5 Under an hour Newbie friendliness 86/100
deepmodeling/LAMBench#443 ·
-
[Code scan] Missing NVE systems crash stability scoring instead of receiving the failure penalty Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
deepmodeling/LAMBench#440 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
deepmodeling/LAMBench#438 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
deepmodeling/LAMBench#436 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
deepmodeling/LAMBench#435 ·
All issues in deepmodeling/LAMBench
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100