Optimize Raster Reader Memory Usage for Hazard Module
维护者通常 3 天内回复
还没有人认领这个 Issue。
评估
调研方向
Start with the HazardXarrayReader class and its get_hazard_kwargs() method, where the raster data is turned into centroid coordinates for the hazard module. The issue leaves open whether the new options should default to filtering out no-data and zero-intensity points, so settle that with the maintainers first. Done means that centroids are built only for the selected coordinates, with the options exposed on the reader and covered by tests.
由索引模型根据 Issue 内容生成。
描述
The current raster readers for the hazard module require building centroids for every point across the total extent of the raster. For high-resolution rasters, this leads to excessive memory requirements.
The construction requires building a Centroid grid for every point of the raster (even for no-data points)
Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
(Add an option to?) Sub-select the dataset to coordinates which are not always NaN (also 0?)
This approach would save substantial RAM and computation time.
Specifically:
- add optional attributes to
HazardXarrayReader(e.g.ignore_novalue_coords(default True) andignore_zero_intensity_coords(default False) - based on these attributes, sub-select the data in the
get_hazard_kwargs()method, to generate Centroids only for the corresponding coordinates.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
- This is related to the refactoring of the centroids (GeoDataFrame)
- This issue affects both read raster and potentially write raster methods? (a priori not)
- 主要语言
- Jupyter Notebook
- 星标
- 478
- 派生
- 166
- 平均合并
- 2 天 31 分钟
- 30 天内合并 PR
- 4
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
CLIMADA-project/climada_python 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 72/100
CLIMADA-project/climada_python#1337 ·
维护者通常 3 天内回复
-
bug
难度 2/5 1-3 小时 新手友好度 85/100
CLIMADA-project/climada_python#1336 ·
维护者通常 3 天内回复
-
CartoDB basemaps now need an API key (somewhat urgent)可能已有人在做 关联的 PR 仍在进行中或已合并。 未关闭bug
难度 2/5 1-3 小时 新手友好度 72/100
CLIMADA-project/climada_python#1332 · 4 条评论 · 1 个 reaction ·
维护者通常 3 天内回复
-
warnings
难度 2/5 1-3 小时 新手友好度 65/100
CLIMADA-project/climada_python#1241 ·
维护者通常 3 天内回复
-
ResourceWarning未关闭warnings
难度 2/5 1-3 小时 新手友好度 62/100
CLIMADA-project/climada_python#1239 ·
维护者通常 3 天内回复
查看 CLIMADA-project/climada_python 的全部 Issue
相似的 Issue
-
request-theme
难度 1/5 1 小时以内 新手友好度 78/100
LizardByte/ThemerrDB#9390 · 1 条评论 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 78/100
talmolab/sleap-roots-training#79 ·
维护者通常 1 天内回复
-
bug
难度 2/5 1-3 小时 新手友好度 66/100
维护者通常 3 天内回复
-
bug downstream-visible good first issue
难度 2/5 1-3 小时 新手友好度 78/100
维护者通常 1 天内回复
-
bug
难度 2/5 1-3 小时 新手友好度 62/100
lincc-frameworks/nested-pandas#579 ·
维护者通常 1 天内回复