pybind/pybind11

[BUG]: Having trouble creating immutable type with `tp_flags |= Py_TPFLAGS_IMMUTABLETYPE`

Open

#4548 opened on Mar 5, 2023

View on GitHub
 (6 comments) (0 reactions) (0 assignees)C++ (14,677 stars) (2,005 forks)batch import
bugenhancementhelp wanted

Description

Required prerequisites

What version (or hash if on master) of pybind11 are you using?

2.10.3

Problem description

I'm trying to bind a C++ class to Python using py::class_<CustomClass>. I have some internal knowledge about my custom class, which should be immutable. So I want to turn on Py_TPFLAGS_IMMUTABLETYPE (since Python 3.10) in my custom Python type object's tp_flags.

I tried to set the tp_flags using py::custom_type_setup:

auto ImmutableCustomTypeObject =
    py::class_<ImmutableCustom>(mod,
                                "ImmutableCustom",
                                "Instances of this class are immutable.",
                                py::custom_type_setup([](PyHeapTypeObject* heap_type) {
                                    auto* type = &heap_type->ht_type;
#ifdef Py_TPFLAGS_IMMUTABLETYPE
                                    type->tp_flags |= Py_TPFLAGS_IMMUTABLETYPE;
#endif
                                }));

However, the typeobject becomes frozen immediately in PyType_Ready after calling the custom_type_setup_callback:

https://github.com/pybind/pybind11/blob/3cc7e4258c15a6a19ba5e0b62a220b1a6196d4eb/include/pybind11/detail/class.h#L716-L735

then a TypeError will raise at line 734 when importing the built extension:

https://github.com/pybind/pybind11/blob/3cc7e4258c15a6a19ba5e0b62a220b1a6196d4eb/include/pybind11/detail/class.h#L733-L735

which is equivalent to:

ImmutableCustom.__module__: str = mod.__name__

it raises:

>>> import _C
TypeError: cannot set '__module__' attribute of immutable type '_C.ImmutableCustom'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: initialization failed

Also, after PyType_Ready, I cannot do cls.def_property_readonly(...).def(...); since it is immutable.


I tried to modify the tp_flags after I had fully initialized the class definition:

auto ImmutableCustomTypeObject =
    py::class_<ImmutableCustom>(mod,
                                "ImmutableCustom",
                                "Instances of this class are immutable.");

reinterpret_cast<PyTypeObject*>(ImmutableCustomTypeObject.ptr())->tp_name = "mymod.ImmutableCustomTypeObject";
py::setattr(ImmutableCustomTypeObject, "__module__", py::str("mymod"));

ImmutableCustomTypeObject
    .def_property_readonly(...)
    .def_property_readonly(...)
    .def(...)
    .def(...);

#ifdef Py_TPFLAGS_IMMUTABLETYPE
reinterpret_cast<PyTypeObject*>(ImmutableCustomTypeObject.ptr())->tp_flags |=
    Py_TPFLAGS_IMMUTABLETYPE;
#endif

I'm not sure if this is allowed to flip tp_flags if the type is already called with PyType_Ready.

Reproducible example code

Initialize a bound class with tp_flags |= Py_TPFLAGS_IMMUTABLETYPE (Python 3.10+):

auto ImmutableCustomTypeObject =
    py::class_<ImmutableCustom>(mod,
                                "ImmutableCustom",
                                "Instances of this class are immutable.",
                                py::custom_type_setup([](PyHeapTypeObject* heap_type) {
                                    auto* type = &heap_type->ht_type;
#ifdef Py_TPFLAGS_IMMUTABLETYPE
                                    type->tp_flags |= Py_TPFLAGS_IMMUTABLETYPE;
#endif
                                }));

Then import the C++ extension in Python:

>>> import _C
TypeError: cannot set '__module__' attribute of immutable type '_C.ImmutableCustom'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: initialization failed

Is this a regression? Put the last known working version here if it is.

Not a regression

Contributor guide