AgentiGrid: parse_matpower 2× faster with identical results (per-character loop and number conversion)
Maintainers usually reply within 3 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 76/100
- Issue type
- Refactor
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- python
- Domain
- performance
Research direction
Run benchmark_parsers.py --case case_ACTIVSg10k.m --buses 100000 --upstream to reproduce the 3.40 s parse, then inspect _extract_section_block and _parse_float in the parser module. Replace the per-character bracket scan with a regex/search and try float() before lower/strip on numbers. Done means the same benchmark reports about 1.76 s and parsed output matches the original case data.
Written by the indexing model from the issue text.
Description
Issue applies to:
- PFLOW
- OPFLOW
- TCOPFLOW
- SCOPFLOW
- SOPFLOW
- CMake and build system
- Spack
- Visualization
- Documentation
- Other
Summary
On a 100,000-bus case (case_ACTIVSg10k.m repeated ten times, 27.5 MB), parse_matpower takes 3.40 s. cProfile shows two hotspots:
_extract_section_block finds a section’s closing bracket with a Python loop over every character (about half the time);
_parse_float lower-cases and strips each of ~5.2 M numbers before float().
| 100,000-bus case | Before | After |
|---|---|---|
Read (parse_matpower) |
3.40 s | 1.76 s (−48 %) |
Write (write_matpower) |
1.01 s | 1.05 s |
| Peak memory while reading | ~299 MB | ~299 MB |
Reproduce: benchmark_parsers.py --case case_ACTIVSg10k.m --buses 100000 --upstream (script and method attached). The change is about 20 lines in one file, and I can open a PR.
Description how to reproduce the issue
No issue, just small performance improvement.
ExaGO version
v2.0.0-12-g43726da1
System and environment details
Local testing
Additional information
OPFLOW on the real case_ACTIVSg10k.m takes 6.95 s wall clock. Reading that case is about 5% of a solve before the change and about 2.5% after; reading OPFLOW’s 2.19 MB console output takes 0.06 s. So this is a small, free improvement, not a bottleneck today. It matters more in sweeps, which read a case per candidate, and on 100k–200k-bus cases.
Proposed solution
A regex finds the brackets, and float() is tried first. The read drops to 1.76 s (−48%). On a second machine it went from 6.35 s to 2.96 s. Write time and memory are unchanged.
- Dominant language
- Python
- Stars
- 17
- Forks
- 11
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 2
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- No Dockerfile or Docker Compose file
- Has a pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ORNL/ExaGO
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
Maintainers usually reply within 3 days
-
[E38] a DC line at the reference bus demotes it to PV: `PSSetUp`'s condition is always trueMay be free again @hambrickjc claimed this 50 days ago, and no pull request is open. Open
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
ORNL/ExaGO#109 · 3 comments · 1 assignee ·
Maintainers usually reply within 3 days
-
bug
Difficulty 3/5 1-2 days Newbie friendliness 45/100
Maintainers usually reply within 3 days
-
bug
Difficulty 3/5 1-2 days Newbie friendliness 68/100
Maintainers usually reply within 3 days
-
Difficulty 4/5 3-5 days Newbie friendliness 55/100
Maintainers usually reply within 3 days
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 73/100
Netcracker/qubership-ai-packages#266 ·
Maintainers usually reply within 1 day
-
PROCESSED_DATA_FOLDER: strip whitespace and treat empty as unsetPossibly taken @TorecLuik claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
NL-BioImaging/BIOMERO.importer#63 ·
Maintainers usually reply within 1 day
-
MLX training-complete chat/merge suggestions omit --backend mlxPossibly taken @vaputa claimed this today. Openneeds-triage
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
MakazhanAlpamys/Soup#2913 · 1 comment ·
Maintainers usually reply within 1 day
-
area:security effort:S priority:P3 status:needs-triage type:chore
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
randomparity/kdive#3657 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
aws/mcp-proxy-for-aws#476 ·
Maintainers usually reply within 1 day