Is it sound to use a slice of numpy::PyReadonlyArray inside pyo3::allow_threads()?
还没有人认领这个 Issue。
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 25/100
调研方向
从 issue 中所示的 ema binding 和 core_ema 入口点开始,然后阅读 PyReadonlyArray 借用文档以及 pyo3::Python::allow_threads 的行为。完成的标准是形成有文档记录且得到项目支持的安全性结论,或形成范围明确的更改或 workaround,并由 maintainers 确定任何相关示例或测试。
由索引模型根据 Issue 内容生成。
描述
Greetings,
My goal is to implement an efficient (and safe) way to wrap a Rust function, using numpy::PyReadonlyArray, with zero-copy and GIL release during computation.
My question is general, but here is a concrete example from my EMA function binding:
use crate::indicators::ema::core_ema;
use numpy::{PyArray1, PyArrayMethods};
use pyo3::pyfunction;
#[pyfunction(signature = (data, window_size = 14, alpha = None))]
pub(crate) fn ema<'py>(
py: pyo3::Python<'py>,
data: numpy::PyReadonlyArray1<'py, f64>,
window_size: usize,
alpha: Option<f64>,
) -> pyo3::PyResult<pyo3::Py<numpy::PyArray1<f64>>> {
let slice = data.as_slice()?;
let py_array_out = PyArray1::<f64>::zeros(py, [len], false);
let py_array_ptr = unsafe { py_array_out.as_slice_mut()? };
py.allow_threads(|| core_ema(slice, window_size, alpha.into(), py_array_ptr))
.map_err(|e| pyo3::exceptions::PyValueError::new_err(format!("{:?}", e)))?;
Ok(py_array_out.into())
}
I want to release the GIL to enable multithreading. I wonder how to do it safely. Or if it's possible.
The documentation of numpy::PyReadonlyArray says:
An instance of this type ensures that there are no instances of PyReadwriteArray, i.e. that only shared references into the interior of the array can be created safely
Additionnally, in the numpy module borrow documentation, it says:
The aim of this module is to ensure that safe Rust code is unable to violate these requirements on its own. We cannot prevent unchecked code - this includes unsafe Rust, Python or other native code like C or Fortran - from violating them. Therefore the responsibility to avoid this lies with the author of that code instead of the compiler
So, in my understanding, I get a reference to input data which is owned by Python and can, in theory, be changed from the outside (e.g. by another Python thread). So in my understanding, the answer should be 'no' it is not sound. But I'm not sure if there is a way, or a workaround or a good practice or if I just need to avoid using allow_thread with PyReadonlyArray1.
Thanks a lot for your time.
- 主要语言
- Rust
- 星标
- 1.4k
- 派生
- 141
- 平均合并
- 16 分钟
- 30 天内合并 PR
- 3
贡献指南
这个仓库没有索引到贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
PyO3/rust-numpy 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 65/100
PyO3/rust-numpy#565 · 1 条评论 ·
-
难度 3/5 1-2 天 新手友好度 58/100
PyO3/rust-numpy#563 ·
-
难度 3/5 1-2 天 新手友好度 56/100
PyO3/rust-numpy#557 ·
-
难度 3/5 1-2 天 新手友好度 45/100
PyO3/rust-numpy#547 · 2 条评论 ·
-
难度 2/5 1-3 小时 新手友好度 20/100
PyO3/rust-numpy#535 ·
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 88/100
-
bug core
难度 2/5 1-3 小时 新手友好度 86/100
-
JIT-compiled number -> Decimal conversion silently overflows instead of raising DECIMAL_OVERFLOW 未关闭fuzz
难度 2/5 1-3 小时 新手友好度 82/100
ClickHouse/ClickHouse#122114 ·
-
难度 1/5 1 小时以内 新手友好度 92/100
linebender/vello_svg#90 ·
-
难度 2/5 1-3 小时 新手友好度 74/100