CI Intermittent `Windows fatal exception: access violation` in `tests/empirical_scope_observation.py`
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 22/100
- Issue 类型
- 缺陷
- 描述清晰度
- 基本清楚
- 活跃度
- 活跃
- 技术栈
- github-actions, numpy, python
- 领域
- testing-qa
调研方向
The crash is raised from blas_math in tests/empirical_scope_observation.py (line 58), run via continuous_integration/run_tests.sh with python -X faulthandler ... blas. Start by reading that test and the threadpoolctl code it drives, then try to reproduce on Windows with the failing environment. Done means identifying which native library faults, typically with a ProcDump capture and WinDbg analysis, or a fix that stops the intermittent access violation in CI.
由索引模型根据 Issue 内容生成。
描述
Partly generated by an agent, but I did double-check with my brain on 😉.
cc @itamarst in case you have some insights or you have seen something similar when putting this empirical tests together.
I may try to investigate further on my Windows VM to see if I can at least reproduce.
The problem
The Windows pylatest_conda_forge_openblas CI job intermittently crashes while running:
PYTHONPATH=. python -X faulthandler tests/empirical_scope_observation.py blas
+ python -X faulthandler tests/empirical_scope_observation.py blas
Windows fatal exception: access violation
Thread 0x00000f48 [ThreadPoolExecutor-0_0Windows fatal exception: ]access violation
(most recent call first):
Current thread's C stack trace (most recent call first):
<cannot get C stack on this system>
File "D:\a\threadpoolctl\threadpoolctl\tests\empirical_scope_observation.py", line 58 in blas_math
File "C:\Users\runneradmin\miniconda3\envs\testenv\Lib\concurrent\futures\thread.py", line 73 in run
File "C:\Users\runneradmin\miniconda3\envs\testenv\Lib\concurrent\futures\thread.py", line 86 in run
File "C:\Users\runneradmin\miniconda3\envs\testenv\Lib\concurrent\futures\thread.py", line 119 in _worker
File "C:\Users\runneradmin\miniconda3\envs\testenv\Lib\threading.py", line 1024 in run
Windows fatal exception: access violation
Current thread's C stack trace (most recent call first):
<cannot get C stack on this system>
./continuous_integration/run_tests.sh: line 34: 1491 Segmentation fault PYTHONPATH=. python -X faulthandler tests/empirical_scope_observation.py blas
Error: Process completed with exit code 139.
Unfortunately, faulthandler doesn't show anything useful on Windows hence the cannot get C stack on this system
Environment
The September 24 failure and September 26 successful run both use:
- Python 3.14.7, with the GIL enabled
- NumPy 2.5.3
- conda-forge OpenBLAS 0.3.34, pthreads build
- The same threadpoolctl commit,
f6bef5dca966327941a87f4a92673cd9c0096add
CI logs
It seems to happen quite often
- 💔 September 22
- 💔 September 24
- 💔 September 25
- ✔️ September 26
- 💔 September 27
To investigate further
It could be useful to capture a native crash dump with ProcDump and inspect it with WinDbg.
LLM suggestion
For this CI crash, use ProcDump to capture a dump, then inspect it with WinDbg. After downloading Microsoft ProcDump, run this in PowerShell with the conda environment activated and the repository as the working directory:
$env:PYTHONPATH = "."
New-Item -ItemType Directory -Force dumps | Out-Null
.\procdump64.exe -accepteula -ma -e 1 -f C0000005 -n 1 `
-x dumps "$env:CONDA_PREFIX\python.exe" `
-X faulthandler tests/empirical_scope_observation.py blas
This captures a full dump at the first access violation (C0000005), before application exception handlers run. Upload dumps/*.dmp as a CI artifact, using if: always() on the upload step. ProcDump options
Open the dump in WinDbg and run:
.symfix
.reload
!analyze -v
.ecxr
kv
~* kb
.ecxr selects the exception context; kv shows its native stack; ~* kb shows every thread’s stack.
Microsoft debugging guidance
Symbol availability determines the detail: without matching symbols, you may get DLL names and offsets rather than internal function names or source lines. Preserve the exact Python, NumPy and OpenBLAS binaries from the failing environment, plus any matching debug symbols. Even an unsymbolized trace can help establish which native library is crashing.
- 主要语言
- Python
- 星标
- 434
- 派生
- 38
- 平均合并
- 2 天 9 小时
- 30 天内合并 PR
- 7
环境准备
这个项目没有提供开发容器、Dockerfile 或贡献指南，环境需要你自己搭建：先看它的 README，通用步骤见我们的新手贡献指南。
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
joblib/threadpoolctl 的其他 Issue
-
难度 4/5 3-5 天 新手友好度 35/100
joblib/threadpoolctl#272 · 7 条评论 ·
维护者通常 1 天内回复
-
难度 4/5 3-5 天 新手友好度 25/100
joblib/threadpoolctl#271 ·
维护者通常 1 天内回复
-
Make it possible to inspect active/spin wait behavior of LLVM/Intel OpenMP via `kmp_get_blocktime`未关闭
难度 3/5 1-2 天 新手友好度 42/100
joblib/threadpoolctl#270 ·
维护者通常 1 天内回复
-
难度 4/5 1-2 天 新手友好度 32/100
joblib/threadpoolctl#264 · 2 条评论 ·
维护者通常 1 天内回复
-
Opt-in API that has a cached `ThreadpoolController`可能已有人在做 @itamarst 于 26 天前认领。 未关闭
难度 4/5 3-5 天 新手友好度 25/100
joblib/threadpoolctl#254 · 3 条评论 ·
维护者通常 1 天内回复
查看 joblib/threadpoolctl 的全部 Issue
相似的 Issue
-
bug documentation
难度 2/5 1-3 小时 新手友好度 78/100
awslabs/visual-asset-management-system#514 ·
维护者通常 1 天内回复
-
backend bug spec:REQ-053
难度 2/5 1-3 小时 新手友好度 66/100
nolte/kamerplanter#2547 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 75/100
EleutherAI/lm-evaluation-harness#4425 ·
维护者通常 1 天内回复
-
Gemini: respect PLANNING_DISABLED=1 in lifecycle context hooks可能已有人在做 @AmariAugustin 今天认领。 未关闭bug good first issue help wanted
难度 2/5 1-3 小时 新手友好度 87/100
OthmanAdi/planning-with-files#329 · 1 条评论 ·
维护者通常 1 天内回复
-
wr.dynamodb.read_items returns [{}] for a single partition value that does not exist可能已有人在做 @auraseb 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 84/100
aws/aws-sdk-pandas#3520 ·
维护者通常 2 天内回复