Null pointer dereference on C API allocation failure in PyArray::new_with_data

Verify that ptr is non-null immediately after calling PyArray_NewFromDescr (and similarly in new_uninit). If ptr.is_null(), ensure that container (or any owned resources) is safely disposed of or converted back to an owned wrapper to prevent memory leaks, and propagate an error or trigger a clean Rust panic before calling PyArray_SetBaseObject or Bound::from_owned_ptr.

Unsafe Rust Review: numpy (v0_28)

Overall Safety Assessment

numpy (v0_28) provides safe Rust bindings and N-dimensional array integration for NumPy via PyO3. The crate architecture relies heavily on dynamically loaded C API function pointers (PY_ARRAY_API and PY_UFUNC_API), type-erased heap containers (PySliceContainer), and global dynamic borrow tracking (BorrowFlags). While the high-level API design aims to encapsulate raw pointers inside transparent PyO3 Bound wrappers and runtime aliasing checks, the codebase exhibits a high density of unsafe operations. The audit identified two critical soundness findings: an unconditional segfault upon Python memory allocation failure (OOM) during array creation, and potential undefined behavior from pointer::offset_from across distinct heap allocations during aliasing conflict checks. Additionally, numerous internal low-level FFI wrappers and pointer helpers lack rigorous safety documentation and proof-obligation annotations.

Critical Findings

1. Null Pointer Dereference / Segfault on Python Memory Allocation Failure in PyArray::new_with_data (src/array.rs:248) 🔴 🚨

• Priority: 🔴 High
• Threat Vector: 🚨 Untrusted Input
• Bug Type: Null Pointer Dereference
• Vulnerability: In PyArray::new_with_data (and similarly in new_uninit), PY_ARRAY_API.PyArray_NewFromDescr(...) is invoked to allocate a new underlying PyArrayObject on the Python heap. If Python runs out of memory (MemoryError) or invalid dimensions are supplied, PyArray_NewFromDescr returns NULL (ptr == 0x0). Immediately following this invocation, without checking whether ptr is non-null, PY_ARRAY_API.PyArray_SetBaseObject(py, ptr as *mut npyffi::PyArrayObject, container as *mut ffi::PyObject) is invoked. In the NumPy C API (arrayobject.c), PyArray_SetBaseObject(arr, obj) directly dereferences arr (arr->base = obj). Passing NULL as arr triggers an immediate null pointer dereference and process crash (segfault / Undefined Behavior).
• Impact: In safe Rust entry points such as IntoPyArray::into_pyarray (which calls from_raw_parts -> new_with_data), allocation failures under low-memory conditions result in immediate undefined behavior and process termination rather than unwinding cleanly or returning a Python memory error.
• Remediation: Verify if ptr.is_null() immediately after calling PyArray_NewFromDescr. If null, ensure container is safely disposed of (or converted back to an owned reference/bound object to prevent leaks) and return an error or trigger a clean panic before calling PyArray_SetBaseObject.

2. Undefined Behavior from pointer::offset_from Across Disjoint Allocations in BorrowKey::conflicts (src/borrow/shared.rs:245) 🟠 🧪

• Priority: 🟠 Medium

• Threat Vector: 🧪 Contrived Setup

• Bug Type: Out-of-Bounds Pointer Arithmetic, Pointer Provenance Violation

• Vulnerability: In BorrowKey::conflicts, to determine whether two aliased NumPy arrays conflict in memory, the byte distance between their data buffers is computed via unsafe { self.data_ptr.offset_from(other.data_ptr).abs() }. According to standard library documentation (std::ptr::offset_from), invoking offset_from is undefined behavior unless both pointers are derived from the exact same allocated object (provenance domain) and their difference is within isize::MAX bytes. While two arrays sharing the same root NumPy base object typically share the same underlying memory buffer, custom C extensions, ctypes/cffi code, or PyArray_SetBaseObject usages can attach the same Python base object to array descriptors pointing to completely disjoint heap allocations. Evaluating offset_from across disjoint allocations triggers undefined behavior.

• Impact: Inspecting or borrowing malformed or custom-backed NumPy arrays from safe Rust code can trigger miscompilations or invalid pointer distance calculations.

• Remediation: Remove the unsafe block entirely and replace offset_from with (self.data_ptr as usize).abs_diff(other.data_ptr as usize). Casting raw pointers to usize and computing their absolute difference calculates the exact byte distance safely without any unsafe block or provenance restrictions.

Fishy Findings

1. Unchecked Return Value and Reference Leak on Failure in PyArray::new_with_data (src/array.rs:271) 🟡 ⚠️

• Priority: 🟡 Low

• Threat Vector: ⚠️ Accidental Misuse

• Bug Type: Resource Leak

• PyArray_SetBaseObject returns c_int (0 on success, -1 on failure), but its return value is discarded. If PyArray_SetBaseObject fails (-1), NumPy C API semantics state it may or may not have stolen the reference to container. Furthermore, container is passed as a raw pointer *mut PyAny derived from container.into_ptr(). If Bound::from_owned_ptr(py, ptr) on line 277 panics, or if PyArray_SetBaseObject fails without stealing the reference, the owned reference count on container is leaked.

2. Fragile FFI Assumption in PyArrayDescr::new (src/dtype.rs:86) Regarding PyArray_DescrConverter2 🟡 ⚠️

• Priority: 🟡 Low

• Threat Vector: ⚠️ Accidental Misuse

• Bug Type: Fragile FFI Assumption

• PyArray_DescrConverter2 is called with &mut descr initialized to null_mut(). If obj cannot be converted, PyArray_DescrConverter2 returns NPY_FAIL (0) and leaves descr null. The code subsequently calls Bound::from_owned_ptr_or_err(py, descr.cast()), which checks if ptr.is_null(). While functional, relying on out-parameter nullness rather than checking the FFI function's explicit return code is error-prone.

3. Broad Scope of unsafe fn Bodies Masking Unannotated Unsafe Operations 🟡 ⚠️

• Priority: 🟡 Low

• Threat Vector: ⚠️ Accidental Misuse

• Bug Type: Missing Unsafe Demarcation

• Across src/array.rs, src/convert.rs, and src/npyffi/, many functions are declared pub unsafe fn or unsafe fn and execute multiple distinct raw FFI calls, pointer offsets, and type casts directly within the top-level function scope. Under Rust 2024 semantics (#[deny(unsafe_op_in_unsafe_fn)]), unsafe operations inside unsafe fn require explicit unsafe {} blocks to clearly demarcate proof obligations from standard logic.

Missing Safety Comments

• src/dtype.rs:55: Missing # Safety doc comment on unsafe impl PyTypeInfo for PyArrayDescr justifying why type descriptor representation satisfies PyO3 type info contracts. 🔴
• src/dtype.rs:61: Missing // SAFETY: comment on PY_ARRAY_API.get_type_object call. 🔴
• src/dtype.rs:87: Missing // SAFETY: comment on PyArray_DescrConverter2 FFI invocation. 🔴
• src/dtype.rs:116: Missing // SAFETY: comment on PyArray_DescrFromType FFI invocation. 🔴
• src/dtype.rs:123: Missing // SAFETY: comment on PyArray_DescrNewFromType FFI invocation. 🔴
• src/dtype.rs:162: Missing // SAFETY: comment on raw pointer dereference &*self.as_dtype_ptr(). 🔴
• src/dtype.rs:187: Missing // SAFETY: comment on raw pointer dereference &*self.as_dtype_ptr(). 🔴
• src/dtype.rs:198: Missing // SAFETY: comment on raw pointer dereference &*self.as_dtype_ptr(). 🔴
• src/dtype.rs:209: Missing // SAFETY: comment on raw pointer dereference &*self.as_dtype_ptr(). 🔴
• src/dtype.rs:326: Missing // SAFETY: comment on PyArray_EquivTypes FFI invocation. 🔴
• src/dtype.rs:333: Missing // SAFETY: comment on &*self.as_dtype_ptr(). 🔴
• src/dtype.rs:334: Missing // SAFETY: comment on PyType::from_borrowed_type_ptr. 🔴
• src/dtype.rs:338: Missing // SAFETY: comment on PyDataType_ELSIZE call. 🔴
• src/dtype.rs:342: Missing // SAFETY: comment on PyDataType_ALIGNMENT call. 🔴
• src/dtype.rs:346: Missing // SAFETY: comment on PyDataType_FLAGS call. 🔴
• src/dtype.rs:350, 353, 358, 361, 368, 373, 380, 384, 391, 403, 406: Missing // SAFETY: comments on internal FFI descriptor accesses and unchecked PyO3 casts. 🔴
• src/dtype.rs:581 (inside impl_element_scalar! macro): Missing // SAFETY: proof comment justifying why primitive scalar types are trivially copyable (IS_COPY = true) and safely managed by NumPy buffers. 🔴
• src/dtype.rs:611: Missing // SAFETY: comment on unsafe impl Element for bf16. 🔴
• src/dtype.rs:636: Missing // SAFETY: comment on unsafe impl Element for Py<PyAny> justifying why PyObject* pointers (IS_COPY = false) are soundly managed in NumPy object arrays. 🔴
• src/array.rs:126: Missing # Safety doc comment on unsafe impl PyTypeInfo for PyArray<T, D>. 🔴
• src/array.rs:131: Missing // SAFETY: comment on get_type_object invocation. 🔴
• src/array.rs:145: Missing // SAFETY: comment on PyArray_Check and cast_unchecked. 🔴
• src/array.rs:223: Missing # Safety doc comment on unsafe fn new_uninit explaining invariants on strides and flag. 🔴
• src/array.rs:248: Missing # Safety doc comment on unsafe fn new_with_data explaining invariants on strides, data_ptr, and container. 🔴
• src/array.rs:280: Missing # Safety doc comment on unsafe fn from_raw_parts. 🔴
• src/array.rs:378: Missing // SAFETY: comment on PyArray_Zeros invocation. 🔴
• src/array.rs:410: Missing // SAFETY: comment on Self::from_raw_parts call. 🔴
• src/array.rs:486: Missing // SAFETY: comment on Self::from_raw_parts call. 🔴
• src/array.rs:514: Missing // SAFETY: comment on uninitialized buffer writing via clone_elements. 🔴
• src/array.rs:690: Missing // SAFETY: comment on PyArray_Arange invocation. 🔴
• src/array.rs:703: Missing # Safety doc comment on unsafe fn clone_elements explaining capacity and alignment invariants on data_ptr. 🔴
• src/array.rs:1341: Missing // SAFETY: comment on raw pointer dereference *self.data(). 🔴
• src/array.rs:1353: Missing // SAFETY: comment on pointer arithmetic slf.data().offset(offset). 🔴
• src/array.rs:1384: Missing // SAFETY: comment on pointer arithmetic data_ptr.offset(...). 🔴
• src/array.rs:1474: Missing // SAFETY: comment on self.cast_unchecked() justifying transparent memory compatibility with PyUntypedArray. 🔴
• src/array.rs:1479: Missing // SAFETY: comment on (*self.as_array_ptr()).data.cast(). 🔴
• src/array.rs:1578, 1588: Missing // SAFETY: comments on RawArrayView::from_shape_ptr calls. 🔴
• src/array.rs:1598: Missing // SAFETY: comment on self.as_array() call. 🔴
• src/array.rs:1608: Missing // SAFETY: comment on PyArray_CopyInto invocation. 🔴
• src/array.rs:1620, 1628: Missing // SAFETY: comments on PyArray_CastToType and cast_into_unchecked. 🔴
• src/array.rs:1639, 1640: Missing // SAFETY: comments on PyArray_Transpose. 🔴
• src/array.rs:1655, 1663: Missing // SAFETY: comments on PyArray_Newshape. 🔴
• src/untyped_array.rs:62: Missing # Safety doc comment on unsafe impl PyTypeInfo for PyUntypedArray. 🔴
• src/untyped_array.rs:67, 71: Missing // SAFETY: comments on FFI type checking. 🔴
• src/untyped_array.rs:135, 161, 171, 176, 200: Missing // SAFETY: comments on raw PyArrayObject flag and dimension inspections. 🔴
• src/untyped_array.rs:229, 262: Missing // SAFETY: comments on slice::from_raw_parts calls for shape and strides. 🔴
• src/untyped_array.rs:296: Missing // SAFETY: comment on Bound::from_borrowed_ptr. 🔴
• src/convert.rs:59, 71: Missing // SAFETY: comments on PyArray::from_raw_parts calls. 🔴
• src/convert.rs:163: Missing // SAFETY: comment on copy_nonoverlapping into uninitialized NumPy array buffer. 🔴
• src/convert.rs:172: Missing // SAFETY: comment on raw pointer writing data_ptr.write. 🔴
• src/convert.rs:202: Missing // SAFETY: comment on nalgebra matrix conversion. 🔴
• src/slice_container.rs:28, 52: Missing # Safety doc comments on drop_boxed_slice and drop_vec. Missing // SAFETY: comments inside on Box::from_raw and Vec::from_raw_parts. 🔴
• src/slice_container.rs:87: Missing // SAFETY: comment on invoking type-erased destructor function pointer (self.drop)(...). 🔴
• src/borrow/mod.rs:282: Missing // SAFETY: comment on self.array.get(index). 🔴
• src/borrow/mod.rs:343: Missing // SAFETY: comment on self.array.try_as_matrix(). 🔴
• src/borrow/shared.rs:42: Missing // SAFETY: comment on unsafe impl Send for Shared. 🔴
• src/borrow/shared.rs:46, 60, 78, 88: Missing # Safety doc comments on extern "C" borrow checking API functions. 🔴
• src/borrow/shared.rs:103, 105: Missing // SAFETY: comments on Send and Sync for SharedPtr. 🔴
• src/borrow/shared.rs:180, 192, 205, 213: Missing // SAFETY: comments on dynamic C API function pointer invocations. 🔴
• src/borrow/shared.rs:372, 376, 387, 398, 405, 427, 434: Missing // SAFETY: comments on raw base object traversing and slice constructions. 🔴
• src/datetime.rs:156, 193: Missing // SAFETY: comments on unsafe impl Element for Datetime and Timedelta. 🔴
• src/datetime.rs:160, 197: Missing // SAFETY: comments on TypeDescriptors::new. 🔴
• src/datetime.rs:218: Missing # Safety doc comment on const unsafe fn new. 🟡
• src/strings.rs:77, 150: Missing // SAFETY: comments on unsafe impl Element for PyFixedString and PyFixedUnicode. 🔴
• src/strings.rs:83, 156: Missing // SAFETY: comments on DTYPES.from_size. 🔴
• src/strings.rs:175: Missing # Safety doc comment on unsafe fn from_size. 🟡
• src/sum_products.rs:70, 128, 152: Missing // SAFETY: comments on PyArray_InnerProduct, PyArray_MatrixProduct, and PyArray_EinsteinSum. 🔴
• src/npyffi/mod.rs:48: Missing // SAFETY: comment on PyArray_GetNDArrayCFeatureVersion. 🔴
• src/npyffi/mod.rs:59, 68, 83: Missing # Safety doc comments and internal // SAFETY: comments on dynamically loaded macro-generated FFI methods. 🔴
• src/npyffi/array.rs:80, 82: Missing // SAFETY: comments on Send and Sync for PyArrayAPI. 🔴
• src/npyffi/array.rs:85: Missing # Safety doc comment on unsafe fn get. 🔴
• src/npyffi/array.rs:375, 388, 410, 463, 469: Missing # Safety doc comments and internal // SAFETY: comments on FFI wrappers. 🔴
• src/npyffi/objects.rs:97, 103, 121, 133: Missing # Safety doc comments and internal // SAFETY: comments on FFI accessor helpers. 🔴
• src/npyffi/ufunc.rs:27, 29: Missing // SAFETY: comments on Send and Sync for PyUFuncAPI. 🔴
• src/npyffi/ufunc.rs:32: Missing # Safety doc comment on unsafe fn get. 🔴