JuliaPy/PyCall.jl

Implicit dependency on NumPy headers not made explicit anywhere

Open

#814 opened on Sep 4, 2020

View on GitHub
 (1 comment) (0 reactions) (0 assignees)Julia (1,510 stars) (187 forks)batch import
help wanted

Description

Hi, I've been on a quest the past two days to figure out why PyCall (through pyjulia) does not turn an Array on the Julia side into a NumPy array on the Python side (and produces a Python list instead). See https://github.com/paulmelis/blender-julia-test/issues/2 for all the gory details in nailing it down, particularly https://github.com/paulmelis/blender-julia-test/issues/2#issuecomment-687354497

The problem turns out to be that the NumPy installation I was using (as part of Blender) did not contain any header files and so npyinitialize in numpy.jl fails. There is an error() raised at that point with message could not read __multiarray_api.h to parse PyArray_API but that apparently never surfaces to the user level. In this case it gets raised further on from NpyArray but gets hidden in the catch part of function PyObject(a::StridedArray{T}) where T<:PYARR_TYPES. The message certainly isn't printed. It's also not mentioned anywhere in the docs that NumPy headers are being parsed (I could only find a reference to this in #38 from 2013).

So please add a note somewhere in the readme that the headers are a requirement for the NumPy installation used and are being parsed (which is quite unexpected). And NumPy initialization silently failing is also pretty suboptimal.

Contributor guide

Tech stack
python
Domain
documentation
Issue type
documentation
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
under 1 hour
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
None
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
80
Research direction
Examine the repository's readme for documentation about dependencies. Locate the file `numpy.jl` to understand the `npyinitialize` function and the error handling. Investigate the `PyObject` function's catch block to see how errors are suppressed. Add a note in the readme about the requirement for NumPy headers and consider improving error propagation.