[BUG]: Document MSVC `std::mutex` ABI hazard with older `msvcp140.dll` and `_DISABLE_CONSTEXPR_MUTEX_CONSTRUCTOR`
Maintainers usually reply within 6 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 68/100
- Issue type
- Documentation
- Clarity
- Mostly clear
- Activity status
- Quiet
- Domain
- documentation
Research direction
Start with the Windows or compiling documentation and review the suggested MSVC std::mutex compatibility text in this issue. Document the _DISABLE_CONSTEXPR_MUTEX_CONSTRUCTOR workaround for affected extension targets, including the CMake and setuptools examples, and explain verification with llvm-objdump; done when the guidance covers the hazard, trade-offs, and expected import-table symbols.
Written by the indexing model from the issue text.
Description
Required prerequisites
- Make sure you've read the documentation. Your issue may be addressed there.
- Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
- Consider asking first in the Gitter chat room or in a Discussion.
What version (or hash if on master) of pybind11 are you using?
3.0.4
Problem description
We recently debugged a real downstream Windows crash where a pybind11-based extension was built with newer MSVC STL headers, but at runtime its std::mutex lock path resolved to an older msvcp140.dll that had been preloaded by another Python wheel.
Microsoft documents _DISABLE_CONSTEXPR_MUTEX_CONSTRUCTOR as the compatibility workaround for this newer-toolset / older-redistributable mismatch. It would be useful for pybind11’s Windows build documentation to mention this for projects distributing MSVC-built binary wheels.
The hazard
With newer VS 2022 MSVC STL headers, std::mutex can use a constexpr zero-initialized layout. The constructor may emit no runtime _Mtx_init_in_situ call; later lock operations call _Mtx_lock from msvcp140.dll.
If another wheel in the same Python process has already loaded an older msvcp140.dll under the normal DLL name, the Windows loader may bind the extension’s _Mtx_lock import to that older runtime. The older runtime may expect the pre-constexpr mutex layout, so it can interpret the newer zero-initialized object incorrectly and crash with an access violation.
In the observed case, the first affected lock happened inside pybind11 import-time internals initialization:
PYBIND11_MODULE_PYINIT
-> pybind11::detail::ensure_internals()
-> pybind11::detail::get_internals()
-> internals_pp_manager::create_pp_content_once()
-> std::lock_guard<std::mutex> lock(pp_set_mutex_)
But the issue class is broader than that one lock: any std::mutex compiled into the extension can be affected if construction and locking are interpreted by incompatible MSVC STL runtime layouts.
Real-world incident
- Incident: metaopt/optree#278
- Downstream fix: metaopt/optree#279
The crash dump showed:
msvcp140!mtx_do_lock+0x74
optree._C!PyInit__C+0xa098
optree._C+0x4e29
optree._C!PyInit__C+0x3d
The faulting address was inside pyarrow’s bundled msvcp140.dll 14.28.29334.0. The affected wheel’s import table showed _Mtx_lock / _Mtx_unlock, but not _Mtx_init_in_situ.
After defining _DISABLE_CONSTEXPR_MUTEX_CONSTRUCTOR for the extension target, the built .pyd import table included:
_Mtx_lock
_Mtx_init_in_situ
_Mtx_unlock
The original reporter confirmed that the patched wheel imports cleanly in the same environment without the previous import-order workaround.
Suggested documentation text
A note like this could live in the Windows / compiling docs:
MSVC
std::mutexcompatibility with oldermsvcp140.dllWindows extension modules built with newer MSVC STL headers and dynamic CRT linking may be loaded into Python processes where another wheel has already loaded an older
msvcp140.dll.If the extension uses
std::mutex, this can produce an MSVC STL mutex ABI mismatch: the extension’s mutex object is constructed using the newer header layout, but_Mtx_lockresolves at runtime to an oldermsvcp140.dllimplementation that expects the older layout. One symptom is an access violation inmsvcp140!mtx_do_lockduring module import.Microsoft provides
_DISABLE_CONSTEXPR_MUTEX_CONSTRUCTORas a compatibility workaround. Projects affected by this can define it for their extension target:if(MSVC) target_compile_definitions(your_module PRIVATE _DISABLE_CONSTEXPR_MUTEX_CONSTRUCTOR) endif()For setuptools:
import sys from setuptools import Extension ext = Extension( 'your_module', sources=[...], define_macros=( [('_DISABLE_CONSTEXPR_MUTEX_CONSTRUCTOR', None)] if sys.platform == 'win32' else [] ), )You can verify that the macro affected the build by inspecting the
.pydimport table:llvm-objdump -p your_module.cp*-win_amd64.pyd | grep _Mtx_Expected output includes
_Mtx_init_in_situalongside_Mtx_lock/_Mtx_unlock._Mtx_destroy_in_situis not expected.Trade-offs: mutex construction is no longer a constexpr no-op, and
constinit std::mutexpatterns are disabled inside that target. For typical extension modules this is usually negligible, but projects should apply the macro at the extension-target level rather than globally.
Reproducible example code
Is this a regression? Put the last known working version here if it is.
Not a regression
- Dominant language
- C++
- Stars
- 18k
- Forks
- 2.3k
- Avg merge
- 6d 13h
- Merged PRs (30d)
- 8
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from pybind/pybind11
-
triage
Difficulty 1/5 Under an hour Newbie friendliness 78/100
Maintainers usually reply within 6 days
-
triage
Difficulty 1/5 Under an hour Newbie friendliness 62/100
pybind/pybind11#5604 · 1 comment ·
Maintainers usually reply within 6 days
-
triage
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
Maintainers usually reply within 6 days
-
docs
Difficulty 2/5 1-3 hours Newbie friendliness 60/100
pybind/pybind11#747 · 2 comments ·
Maintainers usually reply within 6 days
-
Difficulty 3/5 1-2 days Newbie friendliness 55/100
Maintainers usually reply within 6 days
Similar issues
-
agent:WSL bug linux LOW ui
Difficulty 1/5 Under an hour Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Copter: PosHold brake-entry threshold became 16 deg instead of 0.16 deg after the radians conversionOpen
Difficulty 1/5 Under an hour Newbie friendliness 78/100
ArduPilot/ardupilot#34617 · 1 comment · 1 reaction ·
Maintainers usually reply within 1 day
-
bug needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
microsoft/microsoft-ui-xaml#12158 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
Maintainers usually reply within 4 days
-
documentation
Difficulty 1/5 Under an hour Newbie friendliness 65/100
SynthstromAudible/DelugeFirmware#4969 ·
Maintainers usually reply within 1 day