pybind/pybind11

[enhancement] PyPy improvement idea

Open

#2549 opened on Oct 5, 2020

View on GitHub
 (1 comment) (0 reactions) (0 assignees)C++ (14,677 stars) (2,005 forks)batch import
enhancementhelp wanted

Description

A less horribly slow method would be to pre-compile the Python source code, not re-evaluating it from a string every time. Or to expand the Python code into a series of PyXxx() function calls, with something along the lines of PyObject_CallMethod(sys_module, "_getframe", PyLong_FromLong(1)) and some PyObject_GetAttrString() on the result.

I imagine you already know it, but the kind of check done here (for both CPython or PyPy) is quite hackish. I would guess the problem is that when the user writes BaseClassInCpp.method(self) it really invokes the derived method, which makes an infinite loop in the case of overloaded methods in Python that wants to call the parent implementation. It would be an API change, but couldn't BaseClassInCpp.method(self) always invoke the base class method (or the closest written in C++), which might be different from self.method()?

Originally posted by @arigo in https://github.com/pybind/pybind11/issues/2436#issuecomment-685671487

Contributor guide

Tech stack
cpppython
Domain
backendperformance
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
5
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
over 1 week
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
needs investigation
Prerequisites
Familiarity with pybind11 internalsUnderstanding of CPython C APIPyPy internals
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
15
Research direction
Investigate how pybind11 currently handles Python method calls and interacts with PyPy. Explore the feasibility of pre compiling Python source code as suggested in the issue, or expanding Python code into PyXxx() function calls. Refer to the original comment at https://github.com/pybind/pybind11/issues/2436#issuecomment 685671487 and examine the pybind11 source code to understand the current implementation's limitations.