AgentiGrid: parse_matpower 2× faster with identical results (per-character loop and number conversion)

Open Beginner friendly
#158 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 3 days

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
76/100
Issue type
Refactor
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
performance

Research direction

Run benchmark_parsers.py --case case_ACTIVSg10k.m --buses 100000 --upstream to reproduce the 3.40 s parse, then inspect _extract_section_block and _parse_float in the parser module. Replace the per-character bracket scan with a regex/search and try float() before lower/strip on numbers. Done means the same benchmark reports about 1.76 s and parsed output matches the original case data.

Written by the indexing model from the issue text.

Description

Issue applies to:
  • PFLOW
  • OPFLOW
  • TCOPFLOW
  • SCOPFLOW
  • SOPFLOW
  • CMake and build system
  • Spack
  • Visualization
  • Documentation
  • Other
Summary

On a 100,000-bus case (case_ACTIVSg10k.m repeated ten times, 27.5 MB), parse_matpower takes 3.40 s. cProfile shows two hotspots:

_extract_section_block finds a section’s closing bracket with a Python loop over every character (about half the time);
_parse_float lower-cases and strips each of ~5.2 M numbers before float().

100,000-bus case Before After
Read (parse_matpower) 3.40 s 1.76 s (−48 %)
Write (write_matpower) 1.01 s 1.05 s
Peak memory while reading ~299 MB ~299 MB

Reproduce: benchmark_parsers.py --case case_ACTIVSg10k.m --buses 100000 --upstream (script and method attached). The change is about 20 lines in one file, and I can open a PR.

Description how to reproduce the issue

No issue, just small performance improvement.

ExaGO version

v2.0.0-12-g43726da1

System and environment details

Local testing

Additional information

OPFLOW on the real case_ACTIVSg10k.m takes 6.95 s wall clock. Reading that case is about 5% of a solve before the change and about 2.5% after; reading OPFLOW’s 2.19 MB console output takes 0.06 s. So this is a small, free improvement, not a bottleneck today. It matters more in sweeps, which read a case per candidate, and on 100k–200k-bus cases.

Proposed solution

A regex finds the brackets, and float() is tried first. The read drops to 1.76 s (−48%). On a second machine it went from 6.35 s to 2.96 s. Write time and memory are unchanged.

Dominant language
Python
Stars
17
Forks
11
Avg merge
2d 11h
Merged PRs (30d)
2

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

  • No Dockerfile or Docker Compose file
  • Has a pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ORNL/ExaGO

All issues in ORNL/ExaGO

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.