manipulating non-missing elements in a field
还没有人认领这个 Issue。
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 35/100
- Issue 类型
- 功能
- 描述清晰度
- 基本清楚
- 活跃度
- 停滞
- 技术栈
- numpy, python, scikit-learn
- 领域
- data
调研方向
首先检查报告中提到的 CF 字段和 Data 入口点：compressed()、reshape()、get_data() 和 set_data()，包括掩码数组和由 dask 支持的数组是如何处理的。定义用于提取非缺失值和赋值结果的便利操作，然后为 3D-to-2D 工作流以及 dask chunk 大小未知的情况添加覆盖。
由索引模型根据 Issue 内容生成。
描述
In order to use a scikit regression module, I need to
-
Extract the non-missing elements from a 3D field
fin(z,y,x) into a numpy array (z,p), where p is a 1D index over all the non-missing (y,x) locations, which are the same for every z. -
Put a 1D array of results
r(p) into the non-missing data elements of a 2D fieldfout(y,x).
I've done it as follows:
findata=fin.data.compressed().to_dask_array()
findata.compute_chunk_sizes()
findata=findata.reshape(fin.shape[0],fin[0,:].data.compressed().size)
# compute r from findata
foutdata=fout.get_data().flatten()
foutdata[~foutdata.mask]=r
fout.set_data(foutdata.reshape(fout.shape))
Are there better ways, or could you provide better ways? The following occur to me:
- It would be handy if
compressed()was a method of a CF field, to flatten it into a 1D array containing only the non-missing values. - I needed three lines to produce
findatabecause of having tocompute_chunk_sizeson the dask array. Without that step, I got errors as below. - Is there a way to avoid having to get the
Dataout of the field in order to assign to some elements of it using the mask? This was a bit clumsy.
In the Met O IDL lib, both input and output are easier:
findata=fin.data(where(fin(0).data ne fin(0).bmdi))
; compute r from findata
fout.data(where(fout.data ne fout.bmdi))=r
It would be good to have something equally convenient in cf-python. Perhaps there is?
Here's the dask problem:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/users/sws02jmg/.local/lib/python3.11/site-packages/cfdm/decorators.py", line 44, in inplace_wrapper
processed_copy = operation_method(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/users/sws02jmg/.local/lib/python3.11/site-packages/cf/data/data.py", line 10489, in reshape
dx = dx.reshape(*shape, merge_chunks=merge_chunks, limit=limit)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/software/applications/anaconda/2023.09-0/lib/python3.11/site-packages/dask/array/core.py", line 2220, in reshape
return reshape(self, shape, merge_chunks=merge_chunks, limit=limit)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/software/applications/anaconda/2023.09-0/lib/python3.11/site-packages/dask/array/reshape.py", line 222, in reshape
raise ValueError(
ValueError: Array chunk size or shape is unknown. shape: (nan,)
Possible solution with x.compute_chunk_sizes()
Thanks. Jonathan
- 主要语言
- Python
- 星标
- 151
- 派生
- 23
- 平均合并
- 21 小时 29 分钟
- 30 天内合并 PR
- 1
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
NCAS-CMS/cf-python 的其他 Issue
-
Problems creating CFA variables with `cf.write` from aggregation-promoted field ancillaries可能已有人在做 @davidhassell 今天认领。 未关闭aggregation bug dataset read dataset write
难度 2/5 1-3 小时 新手友好度 75/100
-
Remove abandoned `cf.etc` directory可能已有人在做 @sadielbartholomew 于 29 天前认领。 未关闭code tidy
难度 2/5 1-3 小时 新手友好度 76/100
-
installation
难度 4/5 3-5 天 新手友好度 45/100
-
Attempt at unsupported conservative 3D mesh regridding可能已有人在做 @sadielbartholomew 于 26 天前认领。 未关闭bug?
-
Release: migrate from deprecated build command可能已有人在做 @sadielbartholomew 于 29 天前认领。 未关闭release
查看 NCAS-CMS/cf-python 的全部 Issue
相似的 Issue
-
defect from-review v0.9.2
难度 2/5 1-3 小时 新手友好度 86/100
khuisman/mcp-gee-sweet#926 ·
维护者通常 1 天内回复
-
难度 2/5 1 小时以内 新手友好度 84/100
EtanHey/brainlayer#1164 ·
维护者通常 1 天内回复
-
bug security
难度 1/5 1 小时以内 新手友好度 88/100
yunaremaia/vibeguard#141 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 76/100
PrismorSec/prismor#599 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 72/100
维护者通常 1 天内回复