manipulating non-missing elements in a field
還沒有人認領這個 Issue。
評估
- 難度
- 4/5
- 預估耗時
- 3-5 天
- 新手友好度
- 35/100
- Issue 類型
- 功能
- 描述清晰度
- 基本清楚
- 活躍度
- 停滯
- 技術堆疊
- numpy, python, scikit-learn
- 領域
- data
研究方向
先檢查報告中提到的 CF 欄位和 Data 進入點：compressed()、reshape()、get_data() 和 set_data()，包括遮罩陣列和由 dask 支援的陣列如何處理。定義用於擷取非缺失值和指派結果的便利操作，然後為 3D-to-2D 工作流程以及 dask chunk 大小未知的情況新增涵蓋。
由索引模型根據 Issue 內容生成。
描述
In order to use a scikit regression module, I need to
-
Extract the non-missing elements from a 3D field
fin(z,y,x) into a numpy array (z,p), where p is a 1D index over all the non-missing (y,x) locations, which are the same for every z. -
Put a 1D array of results
r(p) into the non-missing data elements of a 2D fieldfout(y,x).
I've done it as follows:
findata=fin.data.compressed().to_dask_array()
findata.compute_chunk_sizes()
findata=findata.reshape(fin.shape[0],fin[0,:].data.compressed().size)
# compute r from findata
foutdata=fout.get_data().flatten()
foutdata[~foutdata.mask]=r
fout.set_data(foutdata.reshape(fout.shape))
Are there better ways, or could you provide better ways? The following occur to me:
- It would be handy if
compressed()was a method of a CF field, to flatten it into a 1D array containing only the non-missing values. - I needed three lines to produce
findatabecause of having tocompute_chunk_sizeson the dask array. Without that step, I got errors as below. - Is there a way to avoid having to get the
Dataout of the field in order to assign to some elements of it using the mask? This was a bit clumsy.
In the Met O IDL lib, both input and output are easier:
findata=fin.data(where(fin(0).data ne fin(0).bmdi))
; compute r from findata
fout.data(where(fout.data ne fout.bmdi))=r
It would be good to have something equally convenient in cf-python. Perhaps there is?
Here's the dask problem:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/users/sws02jmg/.local/lib/python3.11/site-packages/cfdm/decorators.py", line 44, in inplace_wrapper
processed_copy = operation_method(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/users/sws02jmg/.local/lib/python3.11/site-packages/cf/data/data.py", line 10489, in reshape
dx = dx.reshape(*shape, merge_chunks=merge_chunks, limit=limit)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/software/applications/anaconda/2023.09-0/lib/python3.11/site-packages/dask/array/core.py", line 2220, in reshape
return reshape(self, shape, merge_chunks=merge_chunks, limit=limit)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/software/applications/anaconda/2023.09-0/lib/python3.11/site-packages/dask/array/reshape.py", line 222, in reshape
raise ValueError(
ValueError: Array chunk size or shape is unknown. shape: (nan,)
Possible solution with x.compute_chunk_sizes()
Thanks. Jonathan
- 主要語言
- Python
- 星號
- 150
- 分支
- 23
- 平均合併
- 21 小時 29 分鐘
- 30 天內合併 PR
- 1
環境準備
從這裡開始
- 先讀完整個 Issue，再讀專案的貢獻指南。
- 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
- Fork 儲存庫，在一個分支上完成修改。
- 送出 Pull Request，並在描述裡引用這個 Issue 編號。
NCAS-CMS/cf-python 的其他 Issue
-
code tidy
難度 2/5 1-3 小時 新手友好度 76/100
-
Attempt at unsupported conservative 3D mesh regridding可能已有人在做 @sadielbartholomew 於 19 天前認領。 未關閉bug?
-
Release: migrate from deprecated build command可能已有人在做 @sadielbartholomew 於 22 天前認領。 未關閉release
-
Remove support for Python 3.10 (EOL)可能已有人在做 @sadielbartholomew 於 22 天前認領。 未關閉enhancement
-
bug
難度 3/5 1-2 天 新手友好度 45/100
查看 NCAS-CMS/cf-python 的全部 Issue
相似的 Issue
-
bug
難度 2/5 1-3 小時 新手友好度 85/100
維護者通常 1 天內回覆
-
難度 1/5 1 小時以內 新手友好度 90/100
維護者通常 1 天內回覆
-
instance instance add
難度 2/5 1-3 小時 新手友好度 68/100
searxng/searx-instances#941 · 1 則留言 ·
-
難度 1/5 1 小時以內 新手友好度 92/100
FluidNumerics/fluid-walk-blocker#89 ·
維護者通常 1 天內回覆
-
bug
難度 2/5 1-3 小時 新手友好度 84/100
維護者通常 1 天內回覆