pybind/pybind11

[BUG]: Having trouble creating immutable type with `tp_flags |= Py_TPFLAGS_IMMUTABLETYPE`

Open

#4,548 opened on Mar 5, 2023

View on GitHub
 (6 comments) (0 reactions) (0 assignees)C++ (2,005 forks)batch import
bugenhancementhelp wanted

Repository metrics

Stars
 (14,677 stars)
PR merge metrics
 (Avg merge 5d 2h) (10 merged PRs in 30d)

Description

Required prerequisites

What version (or hash if on master) of pybind11 are you using?

2.10.3

Problem description

I'm trying to bind a C++ class to Python using py::class_<CustomClass>. I have some internal knowledge about my custom class, which should be immutable. So I want to turn on Py_TPFLAGS_IMMUTABLETYPE (since Python 3.10) in my custom Python type object's tp_flags.

I tried to set the tp_flags using py::custom_type_setup:

auto ImmutableCustomTypeObject =
    py::class_<ImmutableCustom>(mod,
                                "ImmutableCustom",
                                "Instances of this class are immutable.",
                                py::custom_type_setup([](PyHeapTypeObject* heap_type) {
                                    auto* type = &heap_type->ht_type;
#ifdef Py_TPFLAGS_IMMUTABLETYPE
                                    type->tp_flags |= Py_TPFLAGS_IMMUTABLETYPE;
#endif
                                }));

However, the typeobject becomes frozen immediately in PyType_Ready after calling the custom_type_setup_callback:

https://github.com/pybind/pybind11/blob/3cc7e4258c15a6a19ba5e0b62a220b1a6196d4eb/include/pybind11/detail/class.h#L716-L735

then a TypeError will raise at line 734 when importing the built extension:

https://github.com/pybind/pybind11/blob/3cc7e4258c15a6a19ba5e0b62a220b1a6196d4eb/include/pybind11/detail/class.h#L733-L735

which is equivalent to:

ImmutableCustom.__module__: str = mod.__name__

it raises:

>>> import _C
TypeError: cannot set '__module__' attribute of immutable type '_C.ImmutableCustom'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: initialization failed

Also, after PyType_Ready, I cannot do cls.def_property_readonly(...).def(...); since it is immutable.


I tried to modify the tp_flags after I had fully initialized the class definition:

auto ImmutableCustomTypeObject =
    py::class_<ImmutableCustom>(mod,
                                "ImmutableCustom",
                                "Instances of this class are immutable.");

reinterpret_cast<PyTypeObject*>(ImmutableCustomTypeObject.ptr())->tp_name = "mymod.ImmutableCustomTypeObject";
py::setattr(ImmutableCustomTypeObject, "__module__", py::str("mymod"));

ImmutableCustomTypeObject
    .def_property_readonly(...)
    .def_property_readonly(...)
    .def(...)
    .def(...);

#ifdef Py_TPFLAGS_IMMUTABLETYPE
reinterpret_cast<PyTypeObject*>(ImmutableCustomTypeObject.ptr())->tp_flags |=
    Py_TPFLAGS_IMMUTABLETYPE;
#endif

I'm not sure if this is allowed to flip tp_flags if the type is already called with PyType_Ready.

Reproducible example code

Initialize a bound class with tp_flags |= Py_TPFLAGS_IMMUTABLETYPE (Python 3.10+):

auto ImmutableCustomTypeObject =
    py::class_<ImmutableCustom>(mod,
                                "ImmutableCustom",
                                "Instances of this class are immutable.",
                                py::custom_type_setup([](PyHeapTypeObject* heap_type) {
                                    auto* type = &heap_type->ht_type;
#ifdef Py_TPFLAGS_IMMUTABLETYPE
                                    type->tp_flags |= Py_TPFLAGS_IMMUTABLETYPE;
#endif
                                }));

Then import the C++ extension in Python:

>>> import _C
TypeError: cannot set '__module__' attribute of immutable type '_C.ImmutableCustom'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: initialization failed

Is this a regression? Put the last known working version here if it is.

Not a regression

Contributor guide