Spatial partitioning, sorting and shuffling
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
Research direction
Start with the dask-geopandas spatial partitioning guide, then review the linked xarray.IndexVariable PR and out-of-core index issue. Check the linked Xarray spatial-shuffling PR and determine how space-filling curve values and chunked geometry indexes should fit in xvec. Done means spatially partitioned geometry coordinates and GeometryIndex support the intended out-of-core sorting and shuffling.
Written by the indexing model from the issue text.
Description
When dealing with large sets of geometries it would be nice if we could partition (chunk) the geometry coordinate and the GeometryIndex based on spatial locality (thus requiring spatial sorting or shuffling), like explained for geo-dataframes in dask-geopandas' spatial partitioning user guide.
This would require a good amount of work both here and upstream, though:
- refactor
xarray.IndexVariableto allow dask arrays or other lazy arrays https://github.com/pydata/xarray/pull/8124 (although we can work around this by using regularxarray.Variableobjects for now) - add some sort of generic, out-of-core index in Xarray https://github.com/pydata/xarray/issues/1650, or add our own ad-hoc version here
- compute space-filling curve values from geometries (copied or factored out from dask-geopandas?)
- implement spatial shuffling, probably reusing what is being added in Xarray https://github.com/pydata/xarray/pull/9320
- Dominant language
- Python
- Stars
- 139
- Forks
- 14
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from xarray-contrib/xvec
-
CI
Difficulty 3/5 1-2 days Newbie friendliness 35/100
xarray-contrib/xvec#145 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 58/100
xarray-contrib/xvec#141 · 2 comments ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
xarray-contrib/xvec#129 · 1 comment ·
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
xarray-contrib/xvec#113 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
xarray-contrib/xvec#100 ·
All issues in xarray-contrib/xvec
Similar issues
-
essnmx good first issue
Difficulty 1/5 Under an hour Newbie friendliness 95/100
-
[Feature] 奇物选择添加优先级 Open
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
syfoud/Simulated_Scepter#174 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Giskard-AI/giskard-oss#2840 · 1 comment ·
-
A claim comment carrying the issue number is silently declined while the workflow reports success Openarea: repo bug perceived difficulty: 2
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
yeti-platform/yeti#1380 ·