WSL2 GPU-PV: D3D12 device is removed after allocating about 8 GiB of CPU-visible heaps

Aperta
#42,185 3 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
35/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
cpp, linux

Direzione di ricerca

Start by tracing the WSL GPU-PV allocation path around dxgvmb_send_create_allocation and dxgkio_create_allocation; the report does not identify source files or tests. Reproduce the failure with the supplied C++ program and compare WRITE_BACK with NOT_AVAILABLE and native Windows results. Done means confirming why the 57th CPU-visible heap removes the device and establishing the supported budget behavior or a fix.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

GPU wsl2
Windows Version

Windows 10, 10.0.19045.4291

WSL Version

3.0.1.0

Are you using WSL 1, WSL 2, or WSLC?
  • WSL 2
  • WSL 1
  • WSLC
Kernel Version

6.18.40.1-microsoft-standard-WSL2 (stock)

Distro Version

Docker Desktop's WSL 2 backend, with an x86-64 Arch Linux container (VERSION_ID 20261011.0.609700). The test runs headless.

Other Software
  • Docker Desktop 4.71.0 (225177)
  • WSL Direct3D runtime 1.611.1-81528511 and DXCore 10.0.26100.1-240331-1435.ge-release
  • NVIDIA GeForce RTX 3080 (10DE:2206), Windows driver 32.0.16.1714
  • AMD Radeon(TM) Graphics (1002:164E), Windows driver 32.0.21025.1024
Repro Steps
  1. Save the source below as d3d12-heap-repro.cpp.
  2. In the Linux environment, expose /dev/dxg and the WSL Direct3D libraries. For the Docker setup, the device was passed via --device=/dev/dxg and /usr/lib/wsl/lib and /usr/lib/wsl/drivers were bind-mounted read-only at the same paths. The build used GCC 16.2.1 and Arch's directx-headers package (1:1.721.2-1).
  3. Build and run:
g++ -std=c++20 -O2 -pthread -I/usr/include/wsl/stubs d3d12-heap-repro.cpp -ldl -o d3d12-heap-repro
export LD_LIBRARY_PATH="/usr/lib/wsl/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
./d3d12-heap-repro 10de 64 64 144  # NVIDIA
./d3d12-heap-repro 1002 64 64 144  # AMD

The test calls ID3D12Device::CreateHeap up to 64 times and retains each 144 MiB heap. The heaps use D3D12_HEAP_TYPE_CUSTOM, D3D12_CPU_PAGE_PROPERTY_WRITE_BACK, D3D12_MEMORY_POOL_L0, 4 MiB alignment, and no flags. It uses Direct3D 12 directly, with no Vulkan, Mesa, window, GPU command submission, or resource mapping. The test releases the heaps before exit.

For a WSL control, append na to either command to use D3D12_CPU_PAGE_PROPERTY_NOT_AVAILABLE. For the native Windows control, build the same file in an x64 Visual Studio developer prompt with the Windows SDK:

cl /nologo /EHsc /std:c++20 /O2 d3d12-heap-repro.cpp
d3d12-heap-repro.exe 10de 64 64 144
d3d12-heap-repro.exe 1002 64 64 144
Standalone C++ reproducer
#define INITGUID
#include <unknwn.h>
#ifdef _WIN32
#include <d3d12.h>
#include <dxcore.h>
#else
#include <directx/d3d12.h>
#include <directx/dxcore.h>
#endif
#include <cstdio>
#include <cstdlib>
#include <cstring>
#ifdef _WIN32
#include <windows.h>
#else
#include <dlfcn.h>
#endif
#include <vector>
#include <thread>
#include <chrono>

int main(int argc, char** argv) {
    std::setbuf(stdout, nullptr);
    unsigned vendor = argc > 1 ? std::strtoul(argv[1], nullptr, 16) : 0x10de;
    unsigned iterations = argc > 2 ? std::strtoul(argv[2], nullptr, 10) : 1000;
    unsigned retained = argc > 3 ? std::strtoul(argv[3], nullptr, 10) : 56;
    unsigned heapMiB = argc > 4 ? std::strtoul(argv[4], nullptr, 10) : 144;
    if (iterations > 10000 || retained > 256 || retained == 0 || heapMiB == 0 || heapMiB > 512 || retained * heapMiB > 10240) return 2;
#ifdef _WIN32
    auto dxcore = LoadLibraryW(L"dxcore.dll");
    auto d3d = LoadLibraryW(L"d3d12.dll");
#else
    auto dxcore = dlopen("libdxcore.so", RTLD_NOW | RTLD_LOCAL);
    auto d3d = dlopen("libd3d12.so", RTLD_NOW | RTLD_LOCAL);
#endif
    if (!dxcore || !d3d) return 3;
#ifdef _WIN32
    auto createFactory = reinterpret_cast<HRESULT(WINAPI*)(REFIID, void**)>(GetProcAddress(dxcore, "DXCoreCreateAdapterFactory"));
    auto createDevice = reinterpret_cast<decltype(&D3D12CreateDevice)>(GetProcAddress(d3d, "D3D12CreateDevice"));
#else
    auto createFactory = reinterpret_cast<HRESULT(WINAPI*)(REFIID, void**)>(dlsym(dxcore, "DXCoreCreateAdapterFactory"));
    auto createDevice = reinterpret_cast<decltype(&D3D12CreateDevice)>(dlsym(d3d, "D3D12CreateDevice"));
#endif
    IDXCoreAdapterFactory* factory{};
    HRESULT result = createFactory(IID_IDXCoreAdapterFactory, reinterpret_cast<void**>(&factory));
    if (FAILED(result)) return 4;
    IDXCoreAdapterList* adapters{};
    GUID attribute = DXCORE_ADAPTER_ATTRIBUTE_D3D12_GRAPHICS;
    result = factory->CreateAdapterList(1, &attribute, IID_IDXCoreAdapterList, reinterpret_cast<void**>(&adapters));
    if (FAILED(result)) return 5;
    ID3D12Device* device{};
    for (unsigned i = 0; i < adapters->GetAdapterCount(); ++i) {
        IDXCoreAdapter* adapter{};
        if (FAILED(adapters->GetAdapter(i, IID_IDXCoreAdapter, reinterpret_cast<void**>(&adapter)))) continue;
        DXCoreHardwareID hardware{};
        adapter->GetProperty(DXCoreAdapterProperty::HardwareID, sizeof(hardware), &hardware);
        if (hardware.vendorID == vendor) result = createDevice(adapter, D3D_FEATURE_LEVEL_12_0, IID_ID3D12Device, reinterpret_cast<void**>(&device));
        adapter->Release();
        if (device) break;
    }
    adapters->Release();
    factory->Release();
    if (!device) return 6;
    D3D12_HEAP_DESC description{};
    description.SizeInBytes = static_cast<unsigned long long>(heapMiB) << 20;
    description.Alignment = D3D12_DEFAULT_MSAA_RESOURCE_PLACEMENT_ALIGNMENT;
    description.Properties.Type = D3D12_HEAP_TYPE_CUSTOM;
    description.Properties.CPUPageProperty = argc > 5 && std::strcmp(argv[5], "wc") == 0 ? D3D12_CPU_PAGE_PROPERTY_WRITE_COMBINE : D3D12_CPU_PAGE_PROPERTY_WRITE_BACK;
    if (argc > 5 && std::strcmp(argv[5], "na") == 0) description.Properties.CPUPageProperty = D3D12_CPU_PAGE_PROPERTY_NOT_AVAILABLE;
    if (argc > 6 && std::strcmp(argv[6], "nonresident") == 0) description.Flags = D3D12_HEAP_FLAG_CREATE_NOT_RESIDENT;
    description.Properties.MemoryPoolPreference = D3D12_MEMORY_POOL_L0;
    std::vector<ID3D12Heap*> heaps(retained);
    unsigned passed = 0;
    for (unsigned i = 0; i < iterations; ++i) {
        unsigned slot = i % retained;
        if (heaps[slot]) { heaps[slot]->Release(); heaps[slot] = nullptr; }
        HRESULT before = device->GetDeviceRemovedReason();
        result = device->CreateHeap(&description, IID_ID3D12Heap, reinterpret_cast<void**>(&heaps[slot]));
        if (FAILED(result)) {
            std::printf("FAIL iteration=%u hr=0x%08x before=0x%08x after=0x%08x\n", i, static_cast<unsigned>(result), static_cast<unsigned>(before), static_cast<unsigned>(device->GetDeviceRemovedReason()));
            break;
        }
        ++passed;
        if (i % 100 == 0) std::printf("PROGRESS iteration=%u retained=%u\n", i, retained);
    }
    if (passed == iterations && argc > 7) {
        unsigned seconds = std::strtoul(argv[7], nullptr, 10);
        if (seconds > 30) seconds = 30;
        std::printf("HOLD passed=%u seconds=%u\n", passed, seconds);
        std::this_thread::sleep_for(std::chrono::seconds(seconds));
    }
    for (auto heap : heaps) if (heap) heap->Release();
    device->Release();
    std::printf("FINISH vendor=0x%x passed=%u requested=%u\n", vendor, passed, iterations);
    return passed == iterations ? 0 : 7;
}
Expected Behavior

If the CPU-visible heap budget is exhausted, I would expect CreateHeap to return a recoverable allocation error, such as E_OUTOFMEMORY, and leave the existing device usable. Is there a supported way to query or configure the WSL GPU-PV I/O-space budget for this type of allocation?

Actual Behavior

On both adapters, the first 56 heaps (8,064 MiB total) succeed. The 57th call returns 0x887A0005 (DXGI_ERROR_DEVICE_REMOVED). GetDeviceRemovedReason() reports S_OK before the call and 0x887A0020 (DXGI_ERROR_DRIVER_INTERNAL_ERROR) after it.

  • Using 64 MiB or 128 MiB heaps on NVIDIA also fails immediately after 8,064 MiB of successful allocations.
  • With CPUPageProperty=NOT_AVAILABLE in WSL, both adapters complete all 64 allocations (9,216 MiB). Native Windows also completes all 64 with WRITE_BACK on both adapters.
  • Increasing WSL's RAM setting from 16 GB to 24 GB leaves the observed boundary unchanged.
  • On NVIDIA, 1,000 allocations complete when older heaps are released and at most 56 are retained.

The guest /proc/iomem lists an 8 GiB window for the global DXG device (range c00010000-e0000ffff). Microsoft's GPU-PV documentation says this I/O space is used for CPU-visible allocations. The observed boundary is consistent with exhaustion of that window. The exact reason for device removal needs confirmation.

Diagnostic Logs

Representative output, reproduced on both adapters:

FAIL iteration=56 hr=0x887a0005 before=0x00000000 after=0x887a0020
FINISH vendor=0x10de passed=56 requested=64
FAIL iteration=56 hr=0x887a0005 before=0x00000000 after=0x887a0020
FINISH vendor=0x1002 passed=56 requested=64

Both the WSL na control and the native Windows WRITE_BACK control returned:

FINISH vendor=0x10de passed=64 requested=64
FINISH vendor=0x1002 passed=64 requested=64

Guest kernel messages from the allocation-failure investigation:

misc dxg: dxgk: dxgvmb_send_create_allocation: send_create_allocation failed ffffffb5
misc dxg: dxgk: dxgkio_create_allocation: Ioctl failed: -75

AI assistance was used to prepare this report. The results are from local runs.

Lingua principale
C++
Stelle
34.4k
Fork
1.9k
Merge medio
1g 23h
PR unite (30g)
166

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di microsoft/WSL

Tutte le issue di microsoft/WSL

Issue simili

Altre issue su C++

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.