Support SWMM by reading topology from the companion .inp file

Open
#689 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
48/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Quiet
Tech stack
python
Domain
backend

Research direction

Start with src/modelskill/model/adapters/_inp.py and _res1d.py, especially _simplify_colnames, then inspect the extension constructor tables and the committed tests/testdata/swmm.out and swmm.inp fixtures. Resolve how pollutant columns are named and which SWMM link sections are supported before implementing the required .inp topology path. Done means SWMM loading no longer raises for duplicate quality columns, topology and lengths come from the fixture, and the relevant extension-accounting tests pass.

Written by the indexing model from the issue text.

Description

SWMM results are refused today because the reach connectivity is missing from the .out file. It is not a mikeio1d gap — the data is not in the file, and it is in the companion .inp input file, which sits next to every SWMM result. Everything below was verified against the committed tests/testdata/swmm.out and the swmm.inp in DHI/mikeio1d's testdata.

The topology is in the .inp, and it pairs exactly

swmm.inp is in the same upstream directory our four existing fixtures came from (MIT, commit d937466, recorded in tests/testdata/README.md), so it can be vendored the same way.

swmm.out swmm.inp match
reaches / links 13 13 [CONDUITS] exact
nodes 14 13 [JUNCTIONS] + 1 [OUTFALLS] (18) exact

[CONDUITS] gives Name From Node To Node Length, so it supplies both connectivity and real reach lengths:

[CONDUITS]
;;Name           From Node        To Node          Length     Roughness  ...
;;-------------- ---------------- ---------------- ---------- ----------
1                9                10               400        0.01
4                19               20               200        0.01
...

There is no fallback if the .inp is absent. In the .out alone, every reach reports StartNodeIndex == -1, node xcoord/ycoord are nan, geometry is NodePoint(x=nan, y=nan), chainages is a single -1e-30 sentinel, length is 0, and structures is empty. So no geometric reconstruction is possible — the .inp is required, not optional.

Node and reach timeseries load fine, so only the topology is missing.

Blocker: _simplify_colnames raises on every SWMM node

This has to be fixed before any of the above matters. mikeio1d reports each pollutant as SWMM_NODE_QUAL with no pollutant identity attached, so a node with two pollutants yields two identically named columns:

>>> Res1D("swmm.out").nodes["9"].to_dataframe().columns
[..., 'SWMM_NODE_QUAL:9', 'SWMM_NODE_QUAL:9']

_simplify_colnames in src/modelskill/model/adapters/_res1d.py requires exactly one column per quantity, so it raises:

ValueError: There must be exactly one column per quantity, found [('SWMM_NODE_QUAL:9', 'SWMM_NODE_QUAL'), ('SWMM_NODE_QUAL:9', 'SWMM_NODE_QUAL')]

The same happens for SWMM_LINK_QUAL on reaches.

The only source of the pollutant names is the .inp [POLLUTANTS] section, which from_swmm would already be reading. In the fixture it lists TSS then Lead, and Lead declares a co-pollutant fraction of 0.2 against TSS. The two columns hold 15.720075 and 3.144015 — exactly 0.2×, which confirms column order follows [POLLUTANTS] order in this file.

What is not established is whether that ordering is guaranteed across SWMM versions and models. That decides the naming:

  • SWMM_NODE_QUAL_TSS / SWMM_NODE_QUAL_Lead — useful, but confidently wrong if the order ever differs
  • SWMM_NODE_QUAL_1 / _2 — always correct, tells the user nothing

Worth checking the SWMM binary output format spec before choosing.

Sketch

  • Network.from_swmm(out, inp, *, nodes=None, reaches=None), with inp required.
  • Extend src/modelskill/model/adapters/_inp.py (added for EPANET in #687) with a [CONDUITS] reader. The section reader already handles [SECTION] headers, ;-prefixed comments and whitespace-delimited rows.
  • The reach adapter needs connectivity from the parsed rows rather than from reach.start_node. Res1DReach already accepts the start/end Res1DNode objects and only uses reach.start_node to validate them, and it gained a length override in #687 — so this is a narrower change than it looks.
  • Other SWMM link types ([PUMPS], [ORIFICES], [WEIRS], [OUTLETS]) all put Name From-Node To-Node in their first three columns and carry no length. The fixture has none of them, so supporting them would be unverified; decide whether to read them or refuse a model that uses them.
  • .out moves out of _UNSUPPORTED_EXTENSIONS into its own extension set, and _EXTENSION_CONSTRUCTORS gains the from_swmm mapping. test_every_mikeio1d_extension_is_accounted_for keeps the tables honest.

Related limits

  • SWMM link quantities (SWMM_LINK_FLOW, DEPTH, VELOCITY, CAPACITY, …) sit on single-gridpoint reaches, which produce no breakpoints, so they are unreachable for the same reason EPANET's are — #680. Node quantities are unaffected, so a SWMM network would be usable via NodeObservation from day one.
  • [COORDINATES] gives node x/y, but NetworkNode carries no coordinates today, so that is out of scope.
  • MOUSE and Water Hammer remain refused for lack of any fixture — #686.

Supersedes #688, which claimed this was blocked on a fixture that turned out to exist.

Dominant language
Python
Stars
56
Forks
9
Avg merge
57m
Merged PRs (30d)
3

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from DHI/modelskill

All issues in DHI/modelskill

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.