BindTemporaryResource hangs on new IDMLDevice after prior device removal during operator initializer dispatch
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Idoneità per principianti
- 25/100
- Tipo di issue
- Bug
- Chiarezza
- Da chiarire
- Stato di attività
- Tranquilla
- Stack tecnologico
- cpp
- Ambito
- machine-learning
Direzione di ricerca
Start by running the documented reproduction sequence with DirectML 1.15.4, focusing on device removal during operator initializer dispatch and the later BindTemporaryResource call. Compare it with device removal during inference, which does not reproduce the hang. Done means identifying the cause of the CPU deadlock and verifying that BindTemporaryResource returns on the newly created device.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Summary
IDMLBindingTable::BindTemporaryResource with a non-null buffer deadlocks CPU-side (never returns, no exception, no error HRESULT) on a freshly-created IDMLDevice when a prior IDMLDevice in the same process experienced device removal during operator initializer dispatch.
The deadlock is conditional on the cause of the prior device removal: device removal during inference does NOT cause this hang on the revival device. Only device removal during the initial operator initializer dispatch poisons subsequent device creation in the same process.
Environment
- DirectML.dll version: 1.15.4
- OS: Windows 11 Pro Version 25H2 (OS Build 26200.8117)
- GPU: NVIDIA RTX 4060 Ti 16GB
- GPU driver version: 595.97
- D3D12 feature level: 12 (12_2)
Reproduction Conditions
The deadlock requires all three of the following:
- A prior
IDMLDevicein the same process experienced device removal (GetDeviceRemovedReasonreturnsDXGI_ERROR_DEVICE_HUNG/0x887A0005) during anIDMLOperatorInitializerdispatch - A new
IDMLDeviceis created viaDMLCreateDeviceafter the prior device's removal (the underlyingID3D12Deviceis still alive — only the DML device died) - A binding table is created on the new device for an
IDMLOperatorInitializerwhoseGetBindingProperties().TemporaryResourceSize > 0, andBindTemporaryResourceis called with a non-nullDML_BUFFER_BINDING
When all three hold, BindTemporaryResource does not return. The calling thread blocks indefinitely.
Reproduction Sequence
// 1. First IDMLDevice runs a 2225-op operator initializer.
// The dispatch's ExecuteCommandLists returns success and the wait fence advances,
// but GetDeviceRemovedReason reports 0x887A0005 immediately afterward.
// (Specific model: RAFT optical flow, 2225 ops, tempSz=50,323,456 bytes,
// 8 persistent-buffer Conv ops at persistSz=86,528 each.)
// 2. Create new IDMLDevice on the same (still-alive) ID3D12Device
ComPtr<IDMLDevice> revivalDml;
DMLCreateDevice(d3d12Device.Get(), DML_CREATE_DEVICE_FLAG_NONE,
IID_PPV_ARGS(&revivalDml)); // succeeds
// 3. Build operator initializer for the same 2225 ops
ComPtr<IDMLOperatorInitializer> init;
revivalDml->CreateOperatorInitializer(2225, compiledOps, IID_PPV_ARGS(&init));
DML_BINDING_PROPERTIES bp = init->GetBindingProperties();
// bp.TemporaryResourceSize = 50,323,456
// bp.PersistentResourceSize = 692,224 (8 * 86,528)
// bp.RequiredDescriptorCount = 84
// 4. Allocate fresh resources on revival device
// (NOT reused from prior device — all freshly created)
ComPtr<ID3D12DescriptorHeap> heap = CreateDescriptorHeap(84);
ComPtr<IDMLBindingTable> bt = CreateBindingTable(init.Get(), heap.Get(), 84);
ComPtr<ID3D12Resource> tempBuf = CreateUAVBuffer(50'323'456);
// 5. Bind temporary resource — DEADLOCKS HERE, never returns
DML_BUFFER_BINDING bb = { tempBuf.Get(), 0, 50'323'456 };
DML_BINDING_DESC desc = { DML_BINDING_TYPE_BUFFER, &bb };
bt->BindTemporaryResource(&desc); // CPU hang
- Lingua principale
- C++
- Stelle
- 2.6k
- Fork
- 338
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Guida per i contributori
Nessuna guida per i contributori indicizzata per questo repository
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di microsoft/DirectML
-
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 68/100
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 55/100
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 62/100
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 25/100
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 45/100
Tutte le issue di microsoft/DirectML
Issue simili
-
[CI] Nightly Clang build broken: structured binding captured in a lambda in FileDataSource.cpp Apertabug build
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 91/100
facebookincubator/velox#19194 ·
-
JIT-compiled number -> Decimal conversion silently overflows instead of raising DECIMAL_OVERFLOW Apertafuzz
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
ClickHouse/ClickHouse#122114 ·
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 84/100
-
module/agent platform/macos type/bug/regression
Difficoltà 2/5 1-3 ore Idoneità per principianti 88/100
-
enhancement PyCDE
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100