Some notes on alignment with pangeo-forge-recipes around "keys"
还没有人认领这个 Issue。
评估
- 难度
- 5/5
- 预计耗时
- 一周以上
- 新手友好度
- 25/100
- Issue 类型
- 重构
- 描述清晰度
- 需要澄清
- 活跃度
- 停滞
- 技术栈
- python
调研方向
首先阅读 Xarray Beam keys 文档，以及所引用的 pangeo_forge_recipes/patterns.py 第 70-114 行。将这些抽象与项目在 FilePatterns 中处理索引以及并行写入 Zarr 的方式进行比较。该 issue 未定义具体的更改或验收标准，因此在实现之前需要 maintainer 作出决定。
由索引模型根据 Issue 内容生成。
描述
At today's Pangeo Forge meeting, @alxmrs told us a bit more about Xarray Beam. I'm happy to note that we are aligning a bit around certain abstractions. I'm not proposing to merge or share code or anything at this stage. Just happy to see that we have independently arrived at some similar concepts. Comparing side by side may also suggest improvements.
Xarray Beam
https://xarray-beam.readthedocs.io/en/latest/data-model.html#keys-in-xarray-beam
To keep track of how individual records could be combined into a larger (virtual) dataset, Xarray-Beam defines a Key object. > Key objects consist of:
- offsets: integer offests for chunks from the origin in an immutabledict
- vars: The subset of variables included in each chunk, either as a frozenset, or as None to indicate “all variables”.
Pangeo Forge
We currently call this an "index," although I agree that key is a better name. Our use of indexes is less well documented as it is not really public API but more of an internal thing. We mention it here: https://pangeo-forge.readthedocs.io/en/latest/recipe_user_guide/file_patterns.html#inspect-a-filepattern
The index is its own special type of object used internally by recipes, a
pangeo_forge_recipes.patterns.Index, (which is basically a tuple of one or morepangeo_forge_recipes.patterns.DimIndexobjects).
The code is pretty straightforward and the comments explain things better.
These indexes are used both by FilePatterns (see #31) as well as for parallel writing of Zarr datasets.
cc @cisaacstern
- 主要语言
- Python
- 星标
- 170
- 派生
- 15
- 平均合并
- 23 小时 20 分钟
- 30 天内合并 PR
- 2
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
google/xarray-beam 的其他 Issue
-
`OverflowError` when a chunk exceeds 2 GB可能已有人在做 @chase-dwelle 于 152 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 76/100
google/xarray-beam#196 ·
-
难度 5/5 一周以上 新手友好度 25/100
google/xarray-beam#199 ·
-
难度 4/5 3-5 天 新手友好度 30/100
google/xarray-beam#195 ·
-
难度 5/5 一周以上 新手友好度 30/100
google/xarray-beam#173 ·
-
难度 4/5 3-5 天 新手友好度 35/100
google/xarray-beam#124 ·
查看 google/xarray-beam 的全部 Issue
相似的 Issue
-
bug difficulty:2 good first issue help wanted
难度 2/5 1-3 小时 新手友好度 88/100
MakazhanAlpamys/Soup#1865 ·
维护者通常 1 天内回复
-
area: agents area: docs good first issue
难度 2/5 1-3 小时 新手友好度 88/100
openapi/gknowledge#23 ·
-
useless-suppression reported for deprecated-* messages that should be exempt可能已有人在做 @adityasinha-ghub 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 88/100
pylint-dev/pylint#11656 ·
维护者通常 1 天内回复
-
area:provisioning effort:S priority:P2 provider:local-libvirt risk:daytime-only status:in-progress type:bug
难度 2/5 1-3 小时 新手友好度 78/100
randomparity/kdive#3203 · 2 条评论 ·
维护者通常 1 天内回复
-
get() for StrEnum可能已有人在做 @FeJo5952 今天认领。 未关闭good first issue
难度 2/5 1-3 小时 新手友好度 72/100
collective/icalendar#1921 · 1 条评论 ·
维护者通常 1 天内回复