[ENH]: Add DLPack casters to pybind11 · pybind/pybind11#3858

2022-04-11T16:26:12.000Z

### Required prerequisites - [X] Make sure you've read the [documentation](https://pybind11.readthedocs.io). Your issue may be addressed there. - [X] Search the [issue tracker](https://github.com/pybind/pybind11/issues) and [Discussions](https:/pybind/pybind11/discussions) to verify that this hasn't already been reported. +1 or comment there if it has. - [X] Consider asking first in the [Gitter chat room](https://gitter.im/pybind/Lobby) or in a [Discussion](https:/pybind/pybind11/discussions/new). ### Problem description * Recently, nanobind added a [new type of array caster](https://github.com/wjakob/nanobind/commit/5d92f75fabb439b1172a33627e7d33557f4ed9d1) that uses DLPack to convert to and from array. We would like to backport this caster type to pybind. DLPack is a protocol for quickly and efficiently sharing arrays between programs that are backed by either CPU or GPU memory. For example, with a properly written caster one could transfer between a CUDA GPU Array to a PyTorch Tensor. While this is currently possible with custom caster in PyBind11, it would be nice to have official caster support to make this type of generic array sharing easier. Recent version of Numpy also have native APIs to create Numpy arrays from DLPack. 1. This is not a replacement for the Eigen / Numpy bindings, but a supplement to them. Our current bindings support versions of Numpy that are fairly old. Additionally, we have a lot of convenience functions for dealing with numpy arrays that DLPack likely will not support (reshaping, resizing etc. 2. This would be implemented as custom type casters. To use them, the user would need to specify an optional include (similar to how they already need to for Eigen/Numpy Bindings. 3. The testing suite and much of the code can be ripped from nanobind. Note that there are signficant differences between the type casters though: https://github.com/wjakob/nanobind#api-differences . We should ensure that the ported casters handle errors the pybind11 way.

(12 comments) (0 reactions) (1 assignee)C++ (14,677 stars) (2,005 forks)batch import

enhancementhelp wanted

Description

Required prerequisites

Make sure you've read the documentation. Your issue may be addressed there.
Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
Consider asking first in the Gitter chat room or in a Discussion.

Problem description

Recently, nanobind added a new type of array caster that uses DLPack to convert to and from array. We would like to backport this caster type to pybind. DLPack is a protocol for quickly and efficiently sharing arrays between programs that are backed by either CPU or GPU memory. For example, with a properly written caster one could transfer between a CUDA GPU Array to a PyTorch Tensor. While this is currently possible with custom caster in PyBind11, it would be nice to have official caster support to make this type of generic array sharing easier. Recent version of Numpy also have native APIs to create Numpy arrays from DLPack.

This is not a replacement for the Eigen / Numpy bindings, but a supplement to them. Our current bindings support versions of Numpy that are fairly old. Additionally, we have a lot of convenience functions for dealing with numpy arrays that DLPack likely will not support (reshaping, resizing etc.
This would be implemented as custom type casters. To use them, the user would need to specify an optional include (similar to how they already need to for Eigen/Numpy Bindings.
The testing suite and much of the code can be ripped from nanobind. Note that there are signficant differences between the type casters though: https://github.com/wjakob/nanobind#api-differences . We should ensure that the ported casters handle errors the pybind11 way.

Contributor guide

Tech stack: cpppython
Domain: dataapiperformance
Issue type: feature
Difficulty: 4
Estimated time: over 1 week
Activity status: stale
Clarity: mostly clear
Prerequisites: C++Pythonpybind11 internalsDLPack protocol
Newbie friendliness: 25
Research direction: Study the nanobind commit (5d92f75fabb439b1172a33627e7d33557f4ed9d1) that implements DLPack casters. Understand the differences between nanobind and pybind11 type casters as outlined in the API differences documentation. Examine the existing numpy and Eigen casters in pybind11 (e.g., in include/pybind11/numpy.h) to understand the pattern. The new DLPack caster should be implemented as optional include files, similar to the Eigen bindings. Review the issue comments for any additional design decisions or questions raised.