Handling of misaligned arrays
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Aptitud para principiantes
- 58/100
Línea de trabajo
Comienza reproduciendo el ejemplo del array desalineado e inspecciona PyArrayMethods::as_slice, PyArrayMethods::get y PyReadonlyArray::as_slice, usando el panic reportado como referencia. Comprueba cómo se manejan los arrays no contiguos existentes y si las pruebas relacionadas cubren los arrays de longitud cero. El trabajo debe incluir un resultado seguro para entradas desalineadas, un manejo seguro de arrays de longitud cero y pruebas que cubran ambos casos.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
There are a few methods that produce references to items in Numpy arrays, such as PyArrayMethods::get and PyArrayMethods::as_slice. Rust references are required to be to aligned data, while Numpy arrays aren't (but usually are in practice, unless you try hard). When passing a misaligned array and using a reference-returning method, rust-numpy current invokes UB in safe code by attempting to create the reference.
I'm not sure how practical it is to handle misaligned arrays in all cases. One place where it would be convenient, however, is for zero-length arrays in as_slice. With Pickle 5 and its PickleBuffer objects now the default pickling method from Python 3.14+, Numpy arrays roundtrip through pickle using them, which in the case of an empty array causes the imported array to be produced by (effectively) numpy.frombuffer(bytearray()). Since bytearray() is concerned with bytes, its alignment is only guaranteed as on 1s. I observed Python 3.14 on Linux x86-64 to reliably put the bytearray() singleton buffer on an odd memory address (see python/cpython#140557), which was causing array-extraction failures in my library when trying to implement custom pickling handling when the object happened to be empty.
For example:
use numpy::{PyArray1, PyArrayMethods};
use pyo3::prelude::*;
use pyo3::types::IntoPyDict;
fn main() -> PyResult<()> {
Python::initialize();
Python::attach(|py| {
let array = PyArray1::<u16>::zeros(py, 4, false);
let view = py
.eval(
c"array.view('u1')[1:-1].view('u2')",
None,
Some(&[("array", array)].into_py_dict(py)?),
)?
.cast_into::<PyArray1<u16>>()?;
assert_eq!(view.readonly().as_slice()?, &[0, 0, 0]);
Ok(())
})
}
will panic in debug mode in the as_slice call without returning an error variant:
thread 'main' (30956354) panicked at /Users/jake/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/numpy-0.27.1/src/array.rs:748:16:
unsafe precondition(s) violated: slice::from_raw_parts requires the pointer to be aligned and non-null, and the total size of the slice not to exceed `isize::MAX`
This indicates a bug in the program. This Undefined Behavior check is optional, and cannot be relied on for safety.
stack backtrace:
0: __rustc::rust_begin_unwind
at /rustc/ded5c06cf21d2b93bffd5d884aa6e96934ee4234/library/std/src/panicking.rs:698:5
1: core::panicking::panic_nounwind_fmt::runtime
at /rustc/ded5c06cf21d2b93bffd5d884aa6e96934ee4234/library/core/src/panicking.rs:122:22
2: core::panicking::panic_nounwind_fmt
at /rustc/ded5c06cf21d2b93bffd5d884aa6e96934ee4234/library/core/src/intrinsics/mod.rs:2435:9
3: core::slice::raw::from_raw_parts::precondition_check
at /Users/jake/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/ub_checks.rs:73:21
4: core::slice::raw::from_raw_parts
at /Users/jake/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/ub_checks.rs:78:17
5: numpy::array::PyArrayMethods::as_slice
at /Users/jake/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/numpy-0.27.1/src/array.rs:748:16
6: numpy::borrow::PyReadonlyArray<T,D>::as_slice
at /Users/jake/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/numpy-0.27.1/src/borrow/mod.rs:273:29
7: rust_numpy_test::main::{{closure}}
at ./src/main.rs:16:36
8: pyo3::marker::Python::attach
at /Users/jake/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/pyo3-0.27.2/src/marker.rs:426:9
9: rust_numpy_test::main
at ./src/main.rs:7:5
10: core::ops::function::FnOnce::call_once
at /Users/jake/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/ops/function.rs:250:5
I would be interested in:
PyArrayMethods::as_slicereturning a safeErrvariant if the array is misaligned, not just if it's non-contiguous (though this would potentially be an API break to change the return type)PyArrayMethods::as_slicesafely returning&[]if the length is zero, regardless of the alignment of the internal pointer of the array- any discussion on how to handle data alignment concerns elsewhere in the library
I can easily prepare a patch that addresses the first two points.
- Lenguaje dominante
- Rust
- Estrellas
- 1.4k
- Forks
- 141
- Merge medio
- 16 min
- PR fusionados (30 d)
- 3
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de PyO3/rust-numpy
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 65/100
PyO3/rust-numpy#565 · 1 comentario ·
-
Dificultad 3/5 1-2 días Aptitud para principiantes 58/100
PyO3/rust-numpy#563 ·
-
Dificultad 3/5 1-2 días Aptitud para principiantes 56/100
PyO3/rust-numpy#557 ·
-
Dificultad 3/5 1-2 días Aptitud para principiantes 45/100
PyO3/rust-numpy#547 · 2 comentarios ·
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 20/100
PyO3/rust-numpy#535 ·
Todos los issues de PyO3/rust-numpy
Issues similares
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 88/100
nautechsystems/nautilus_trader#5095 ·
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
stellar/rs-soroban-env#1739 ·
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 76/100
-
bug good first issue package: quic
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100