pybind/pybind11

[BUG]: Python atexit broken with pybind11 version >= 2.9.2

Open

#4459 opened on Jan 17, 2023

View on GitHub
 (6 comments) (0 reactions) (0 assignees)C++ (14,677 stars) (2,005 forks)batch import
bughelp wanted

Description

Required prerequisites

What version (or hash if on master) of pybind11 are you using?

2.10.2

Problem description

We want to upgrade from pybind11 release 2.9.0 to 2.10.2, but we encountered an issue in the Python atexit (https://docs.python.org/3/library/atexit.html) functions, that our custom types are not registered anymore. With delta debugging we found that this PR caused the issue: https://github.com/pybind/pybind11/pull/3744

In my opinion the

    detail::get_local_internals().registered_types_cpp.clear();    
    detail::get_local_internals().registered_exception_translators.clear();

calls should be after the Py_Finalize function call.

When I change the orders of those function calls, the problem is gone.

Reproducible example code

No response

Is this a regression? Put the last known working version here if it is.

2.9.1

Contributor guide

Tech stack
pythoncpp
Domain
backend
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
4
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
needs maintainer response
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
C++ knowledgepybind11 internalsPython C API
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
50
Research direction
Examine the pybind11 source code, particularly the finalization logic in pybind11/detail/internals.h or pybind11/detail. The bug is that registered types cpp and registered exception translators are cleared before Py Finalize, causing atexit handlers to fail. Move those clears after Py Finalize. Test by creating a pybind11 module that uses atexit and ensure no errors.