Why do CUDA device allocations appear to produce a nearly 1:1 increase in WSL host committed memory? Is this expected behavior related to CUDA/WSL2 GPU memory management, or an actual memory-accounting issue?
I maintainer di solito rispondono entro 1 giorno
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Idoneità per principianti
- 25/100
- Tipo di issue
- Bug
- Chiarezza
- Abbastanza chiara
- Stato di attività
- Attiva
- Stack tecnologico
- python, pytorch
- Ambito
- infrastructure, operating-systems
Direzione di ricerca
Start with the minimal PyTorch reproduction in the issue and compare its CUDA allocation steps with the reported Windows vmmemWSL commit measurements. The report points to CUDA allocation backing or reservation in WSL2 GPU paravirtualization, but names no source files or tests. First determine whether this commit growth is expected; done means documenting the behavior or identifying a reproducible accounting defect.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Windows Version
Microsoft Windows [Version 10.0.26200.7840]
WSL Version
3.0.1.0
Are you using WSL 1, WSL 2, or WSLC?
- WSL 2
- WSL 1
- WSLC
Kernel Version
6.18.40.1-microsoft-standard-WSL2
Distro Version
Ubuntu 24.04.3 LTS (Noble Numbat)
Other Software
Hardware:
- Primary GPU: NVIDIA RTX PRO 6000 Blackwell Workstation Edition (96 GB VRAM, sm_120)
- Secondary GPU: NVIDIA GeForce RTX 4090 (24 GB VRAM)
- NVIDIA Windows Driver: 617.42
- System RAM: 128 GiB
Software:
- Windows 11, Build 26200.7840
- WSL: 3.0.1.0
- WSL2 Kernel: 6.18.40.1-microsoft-standard-WSL2
- Ubuntu 24.04.3 LTS (Noble Numbat)
- PyTorch: 2.9.0+cu128 (verified in native Windows environment; WSL environment version not yet confirmed)
- CUDA runtime (Windows PyTorch): 12.8
WSL2 configuration (%UserProfile%\.wslconfig):
[wsl2]
networkingMode=nat
guiApplications=false
memory=64GB
swap=32GB
[experimental]
autoMemoryReclaim=gradual
The issue occurs during CUDA GPU allocations under WSL2 and can be reproduced using a minimal PyTorch script without a full model or training framework.
Repro Steps
Steps to reproduce
1. Configure WSL2
Configure .wslconfig as described above, then restart WSL using:
wsl --shutdown
Launch Ubuntu and activate a Linux Python environment with CUDA-enabled PyTorch installed.
2. Record initial WSL memory information
Run from Windows PowerShell:
wsl -d Ubuntu -- sh -c 'grep -E "^(MemTotal|MemFree|MemAvailable|Cached|SwapTotal|SwapFree|Committed_AS):" /proc/meminfo'
Observed WSL memory statistics:
MemTotal: 65845096 kB
MemFree: 37435408 kB
MemAvailable: 50050264 kB
Cached: 11298936 kB
SwapTotal: 33554432 kB
SwapFree: 33554432 kB
Committed_AS: 19508596 kB
3. Run a Windows-side commit monitor
Open a separate Windows PowerShell terminal:
while ($true) {
$vm = Get-Process vmmemWSL -ErrorAction SilentlyContinue
$sys = (Get-Counter '\Memory\Committed Bytes').CounterSamples[0].CookedValue
"{0:HH:mm:ss} System={1:N2}GiB WSLCommit={2:N2}GiB WSLWorkingSet={3:N2}GiB" -f `
(Get-Date),
($sys / 1GB),
($vm.PrivateMemorySize64 / 1GB),
($vm.WorkingSet64 / 1GB)
Start-Sleep -Seconds 3
}
This monitors:
- Total Windows system committed memory
vmmemWSLprivate committed memoryvmmemWSLworking set
4. Execute the minimal CUDA allocation test inside Ubuntu WSL2
Create repro_cuda_commit.py:
import gc
import time
import torch
print("GPU:", torch.cuda.get_device_name(0), flush=True)
chunks = []
def measure(stage):
torch.cuda.synchronize()
print(
f"{stage}: GPU allocated="
f"{torch.cuda.memory_allocated()/1024**3:.2f} GiB",
flush=True
)
with open("/proc/meminfo") as f:
for line in f:
if line.startswith(("MemAvailable:", "Committed_AS:")):
print(line.strip(), flush=True)
time.sleep(15)
measure("CUDA initialized")
for i in range(3):
x = torch.empty(
4 * 1024**3,
device="cuda:0",
dtype=torch.uint8
)
x.fill_(1)
chunks.append(x)
measure(f"{(i+1)*4} GiB GPU allocation")
chunks.clear()
del x
gc.collect()
torch.cuda.empty_cache()
measure("GPU allocations released")
Run:
python repro_cuda_commit.py
5. Observe Windows host commit
While the script executes, observe the WSLCommit column in the Windows PowerShell monitor.
GPU memory is allocated in three consecutive 4 GiB increments (4, 8, and 12 GiB total), followed by complete release.
Compare the increase in GPU VRAM allocation against the corresponding increase in vmmemWSL private committed memory.
6. Check Windows page file usage
Get-CimInstance Win32_PageFileUsage |
Format-Table Name, AllocatedBaseSize, CurrentUsage, PeakUsage
This helps distinguish Windows commit accounting from actual page file occupancy.
Expected Behavior
CUDA device-only allocations (torch.empty(..., device="cuda:0")) should primarily consume GPU device memory, without requiring an approximately equal increase in Windows host committed memory unless this is an intentional, documented requirement of WSL2 GPU virtualization.
Specifically:
- Allocating 4 GiB of GPU VRAM should not unexpectedly require approximately 4 GiB of additional Windows system commit.
- Large GPU workloads should not encounter host commit-limit pressure solely because GPU VRAM allocations are being mirrored into Windows commit accounting, unless such accounting is required for the virtualization architecture.
- The relationship between device VRAM allocations,
vmmemWSLprivate committed memory, and Windows commit limits should be documented if this behavior is expected. - Users should not need to continuously increase the Windows page file size simply to accommodate GPU allocations without a corresponding explanation of the host commit requirement.
The goal is to understand whether this near-1:1 host commit growth is expected WSL2/CUDA virtualization behavior or an unnecessary memory-accounting overhead that can be corrected.
Actual Behavior
CUDA device allocations produce approximately 1:1 Windows host commit growth
Running the minimal reproduction on the NVIDIA RTX PRO 6000 Blackwell under WSL2 produces the following measurements:
GPU VRAM allocated | vmmemWSL private commit | Increase from baseline -- | -- | -- 0 GiB | 8.99 GiB | — 4 GiB | 13.12 GiB | +4.13 GiB 8 GiB | 17.13 GiB | +8.14 GiB 12 GiB | 21.10 GiB | +12.11 GiB Released | 9.28 GiB | +0.29 GiBFindings
1. Nearly exact 1:1 relationship
Every additional 4 GiB of CUDA device allocation produces approximately 4 GiB of additional vmmemWSL private committed memory.
At 12 GiB of GPU allocations, WSL host commit has increased by 12.11 GiB.
This strongly suggests the GPU allocation size is directly associated with the additional Windows host commit charge.
2. Memory commitment is reversible
After executing:
chunks.clear()
del x
gc.collect()
torch.cuda.empty_cache()WSL committed memory falls from 21.10 GiB to 9.28 GiB, close to the initial 8.99 GiB baseline.
This makes the behavior reproducible and allocation-dependent rather than evidence of a monotonically growing memory leak.
3. Windows page file occupancy remains low
Separately observed Windows page file statistics:
Name AllocatedBaseSize CurrentUsage PeakUsage
---- ----------------- ------------ ---------
C:\pagefile.sys 32768 468 850All sizes are in MiB.
The page file has 32 GiB allocated, but only approximately 468 MiB was in use at the time of measurement. This illustrates that the observed increase in committed memory is not equivalent to an equal increase in actual page file occupancy.
4. Impact during model training
During larger PyTorch model training workloads on the RTX PRO 6000 Blackwell, total Windows system committed memory has exceeded 70 GB.
The controlled test demonstrates that CUDA VRAM allocations contribute directly to vmmemWSL host commit.
This reduces available Windows commit headroom and can make the system dependent on larger page file configurations even when the additional memory is not actively being paged to disk.
Suspected area
The measurements suggest an interaction between CUDA GPU device allocations, WSL2 GPU virtualization, and Windows host commit accounting.
Possible areas for investigation include CUDA allocation backing or reservation behavior in the WSL2 GPU paravirtualization path.
The test does not establish whether physical host RAM is duplicated or whether the additional charge represents reserved virtual memory.
Question: Is the approximately 1:1 host commit charge for CUDA device allocations an intentional part of WSL2's GPU memory model, or can it be eliminated or reduced?
Diagnostic Logs
System information
Windows Version: 10.0.26200.7840
WSL Version: 3.0.1.0
WSL2 Kernel: 6.18.40.1-microsoft-standard-WSL2
Distribution: Ubuntu 24.04.3 LTS
GPU 0: NVIDIA RTX PRO 6000 Blackwell Workstation Edition
GPU 1: NVIDIA GeForce RTX 4090
NVIDIA Driver: 617.42
Windows PyTorch: 2.9.0+cu128
Windows PyTorch CUDA: 12.8
Windows page file information
Name AllocatedBaseSize CurrentUsage PeakUsage
---- ----------------- ------------ ---------
C:\pagefile.sys 32768 468 850
Controlled GPU allocation results
GPU allocated WSL private commit Delta
0 GiB 8.99 GiB Baseline
4 GiB 13.12 GiB +4.13 GiB
8 GiB 17.13 GiB +8.14 GiB
12 GiB 21.10 GiB +12.11 GiB
Released 9.28 GiB +0.29 GiB
Related issue
This report isolates a different observable behavior from the related CUDA context overhead issue: proportional Windows host commit growth during explicit CUDA device allocations, independently of model loading or inference frameworks.
The minimal PyTorch reproduction is included in the Repro Steps section.
Additional WSL-side PyTorch version information and WSL diagnostic logs can be provided if requested.
- Lingua principale
- C++
- Stelle
- 34.4k
- Fork
- 1.9k
- Merge medio
- 1g 23h
- PR unite (30g)
- 166
Preparare l'ambiente
- Nessun Dockerfile né file Docker Compose
- Ha un modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di microsoft/WSL
-
feature wsl2
Difficoltà 2/5 1-3 ore Idoneità per principianti 80/100
microsoft/WSL#42306 · 3 commenti ·
I maintainer di solito rispondono entro 1 giorno
-
feature
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
microsoft/WSL#42114 · 3 commenti ·
I maintainer di solito rispondono entro 1 giorno
-
feature kernel wsl2
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
microsoft/WSL#41107 · 3 commenti · 1 reazione ·
I maintainer di solito rispondono entro 1 giorno
-
feature kernel
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
microsoft/WSL#41054 · 9 commenti · 1 reazione ·
I maintainer di solito rispondono entro 1 giorno
-
feature GPU
Difficoltà 1/5 1-3 ore Idoneità per principianti 70/100
microsoft/WSL#13592 · 4 commenti ·
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di microsoft/WSL
Issue simili
-
needs-format
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
Niko1221/Strata#6251 · 1 commento ·
I maintainer di solito rispondono entro 5 giorni
-
[Bug]: NP 管理器文件编辑器里删除键无效Apertabug
Difficoltà 2/5 1-3 ore Idoneità per principianti 65/100
metasequoiaime/msime#11666 ·
I maintainer di solito rispondono entro 1 giorno
-
DAE2 crashes on call.without.effects with a non ref.func expressionForse già presa @stevenfontanella l’ha presa oggi. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
WebAssembly/binaryen#9404 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 65/100
CodeZombie/goodluckOS#178 ·
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 66/100
CVO-Org/immersive-cigs-rewrite#313 ·
I maintainer di solito rispondono entro 1 giorno