Fast area-subsetting when loading dfs
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- python
- Domain
- data, performance
Research direction
Start by reading the mikeio.read() area-subsetting path and the mikeio.generic.extract() entry point, then compare how each handles DFS2 data and time ranges. The requested behavior is area-based extraction without loading the full global dataset; done means a small-area, all-timestep extraction completes efficiently and preserves the requested output.
Written by the indexing model from the issue text.
Description
I have a request for fast loading of subset of dfs data.
Background: I have a dfs2 file for a global dataset, and I wish to read all timesteps of this data but only a sub-area. The way I do it now, is to pass a bounding box to the mikeio.read() method:
ds = mikeio.read(fn,area = bbox_tuple)
The problem is that this method is really slow, even thought the area is very small. It seems that MIKEIO needs to load the entire area under the hood before subsetting. For reference: it takes 16 minutes to load the data, even though the resulting dfs2 file is only 1900 kb (the original global file is around 110 GB).
Ideally, it would be possible to use the mikeio.generic.extract() method with an 'area' argument instead of only subsetting in time. I imagine something along the lines of:
mikeio.generic.extract(fn, fn_out, area=bbox_tuple, start=0, end=-1, step=1, items=None)
- Dominant language
- Python
- Stars
- 5
- Forks
- 1
- Avg merge
- 13m
- Merged PRs (30d)
- 3
Getting set up
We have not checked this project's setup files yet. Start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from DHI/mikecore-python
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
DHI/mikecore-python#49 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 45/100
DHI/mikecore-python#59 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 42/100
DHI/mikecore-python#54 · 2 comments ·
-
Difficulty 5/5 Over a week Newbie friendliness 45/100
DHI/mikecore-python#52 ·
-
Remaining TODOs in production code: platform notes, licensing questions, and one likely-stale markerOpen
Difficulty 5/5 Over a week Newbie friendliness 30/100
DHI/mikecore-python#51 ·
All issues in DHI/mikecore-python
Similar issues
-
needs triage
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 2 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
openvinotoolkit/openvino_notebooks#3665 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
Maintainers usually reply within 1 day
-
docs
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Maintainers usually reply within 1 day
-
benchmark-gap
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day