CI Intermittent `Windows fatal exception: access violation` in `tests/empirical_scope_observation.py`
Maintainer antworten meist innerhalb von 1 Tag
Dieses Issue hat noch niemand übernommen.
Bewertung
- Schwierigkeit
- 4/5
- Geschätzter Aufwand
- 3-5 Tage
- Anfängerfreundlichkeit
- 22/100
- Issue-Typ
- Bug
- Klarheit
- Größtenteils klar
- Aktivitätsstatus
- Aktiv
- Tech-Stack
- github-actions, numpy, python
- Bereich
- testing-qa
Rechercherichtung
The crash is raised from blas_math in tests/empirical_scope_observation.py (line 58), run via continuous_integration/run_tests.sh with python -X faulthandler ... blas. Start by reading that test and the threadpoolctl code it drives, then try to reproduce on Windows with the failing environment. Done means identifying which native library faults, typically with a ProcDump capture and WinDbg analysis, or a fix that stops the intermittent access violation in CI.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Beschreibung
Partly generated by an agent, but I did double-check with my brain on 😉.
cc @itamarst in case you have some insights or you have seen something similar when putting this empirical tests together.
I may try to investigate further on my Windows VM to see if I can at least reproduce.
The problem
The Windows pylatest_conda_forge_openblas CI job intermittently crashes while running:
PYTHONPATH=. python -X faulthandler tests/empirical_scope_observation.py blas
+ python -X faulthandler tests/empirical_scope_observation.py blas
Windows fatal exception: access violation
Thread 0x00000f48 [ThreadPoolExecutor-0_0Windows fatal exception: ]access violation
(most recent call first):
Current thread's C stack trace (most recent call first):
<cannot get C stack on this system>
File "D:\a\threadpoolctl\threadpoolctl\tests\empirical_scope_observation.py", line 58 in blas_math
File "C:\Users\runneradmin\miniconda3\envs\testenv\Lib\concurrent\futures\thread.py", line 73 in run
File "C:\Users\runneradmin\miniconda3\envs\testenv\Lib\concurrent\futures\thread.py", line 86 in run
File "C:\Users\runneradmin\miniconda3\envs\testenv\Lib\concurrent\futures\thread.py", line 119 in _worker
File "C:\Users\runneradmin\miniconda3\envs\testenv\Lib\threading.py", line 1024 in run
Windows fatal exception: access violation
Current thread's C stack trace (most recent call first):
<cannot get C stack on this system>
./continuous_integration/run_tests.sh: line 34: 1491 Segmentation fault PYTHONPATH=. python -X faulthandler tests/empirical_scope_observation.py blas
Error: Process completed with exit code 139.
Unfortunately, faulthandler doesn't show anything useful on Windows hence the cannot get C stack on this system
Environment
The September 24 failure and September 26 successful run both use:
- Python 3.14.7, with the GIL enabled
- NumPy 2.5.3
- conda-forge OpenBLAS 0.3.34, pthreads build
- The same threadpoolctl commit,
f6bef5dca966327941a87f4a92673cd9c0096add
CI logs
It seems to happen quite often
- 💔 September 22
- 💔 September 24
- 💔 September 25
- ✔️ September 26
- 💔 September 27
To investigate further
It could be useful to capture a native crash dump with ProcDump and inspect it with WinDbg.
LLM suggestion
For this CI crash, use ProcDump to capture a dump, then inspect it with WinDbg. After downloading Microsoft ProcDump, run this in PowerShell with the conda environment activated and the repository as the working directory:
$env:PYTHONPATH = "."
New-Item -ItemType Directory -Force dumps | Out-Null
.\procdump64.exe -accepteula -ma -e 1 -f C0000005 -n 1 `
-x dumps "$env:CONDA_PREFIX\python.exe" `
-X faulthandler tests/empirical_scope_observation.py blas
This captures a full dump at the first access violation (C0000005), before application exception handlers run. Upload dumps/*.dmp as a CI artifact, using if: always() on the upload step. ProcDump options
Open the dump in WinDbg and run:
.symfix
.reload
!analyze -v
.ecxr
kv
~* kb
.ecxr selects the exception context; kv shows its native stack; ~* kb shows every thread’s stack.
Microsoft debugging guidance
Symbol availability determines the detail: without matching symbols, you may get DLL names and offsets rather than internal function names or source lines. Preserve the exact Python, NumPy and OpenBLAS binaries from the failing environment, plus any matching debug symbols. Even an unsymbolized trace can help establish which native library is crashing.
- Vorherrschende Sprache
- Python
- Sterne
- 434
- Forks
- 38
- Ø Merge
- 2 T. 9 Std.
- Gemergte PRs (30 T.)
- 7
Entwicklungsumgebung
Dieses Projekt bietet weder Dev-Container noch Dockerfile noch Beitragsleitfaden – die Einrichtung liegt bei Ihnen. Beginnen Sie mit der README; die allgemeinen Schritte stehen in unserem Leitfaden für den ersten Beitrag.
Erste Schritte
- Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
- Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
- Forken Sie das Repository und arbeiten Sie in einem Branch.
- Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.
Mehr aus joblib/threadpoolctl
-
Schwierigkeit 4/5 3-5 Tage Anfängerfreundlichkeit 35/100
joblib/threadpoolctl#272 · 7 Kommentare ·
Maintainer antworten meist innerhalb von 1 Tag
-
Make it possible to set active/spin wait behavior of LLVM/Intel OpenMP via `kmp_set_blocktime`Offen
Schwierigkeit 4/5 3-5 Tage Anfängerfreundlichkeit 25/100
joblib/threadpoolctl#271 ·
Maintainer antworten meist innerhalb von 1 Tag
-
Make it possible to inspect active/spin wait behavior of LLVM/Intel OpenMP via `kmp_get_blocktime`Offen
Schwierigkeit 3/5 1-2 Tage Anfängerfreundlichkeit 42/100
joblib/threadpoolctl#270 ·
Maintainer antworten meist innerhalb von 1 Tag
-
Schwierigkeit 4/5 1-2 Tage Anfängerfreundlichkeit 32/100
joblib/threadpoolctl#264 · 2 Kommentare ·
Maintainer antworten meist innerhalb von 1 Tag
-
Opt-in API that has a cached `ThreadpoolController`Evtl. vergeben @itamarst hat das vor 27 Tagen übernommen. Offen
Schwierigkeit 4/5 3-5 Tage Anfängerfreundlichkeit 25/100
joblib/threadpoolctl#254 · 3 Kommentare ·
Maintainer antworten meist innerhalb von 1 Tag
Alle Issues in joblib/threadpoolctl
Ähnliche Issues
-
bug documentation
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 78/100
awslabs/visual-asset-management-system#514 ·
Maintainer antworten meist innerhalb von 1 Tag
-
backend bug spec:REQ-053
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 66/100
nolte/kamerplanter#2547 ·
Maintainer antworten meist innerhalb von 1 Tag
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 75/100
EleutherAI/lm-evaluation-harness#4425 ·
Maintainer antworten meist innerhalb von 1 Tag
-
Gemini: respect PLANNING_DISABLED=1 in lifecycle context hooksEvtl. vergeben @AmariAugustin hat das heute übernommen. Offenbug good first issue help wanted
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 87/100
OthmanAdi/planning-with-files#329 · 1 Kommentar ·
Maintainer antworten meist innerhalb von 1 Tag
-
wr.dynamodb.read_items returns [{}] for a single partition value that does not existEvtl. vergeben @auraseb hat das heute übernommen. Offen
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 84/100
aws/aws-sdk-pandas#3520 ·
Maintainer antworten meist innerhalb von 2 Tagen