Dynamic VRAM (comfy-aimdo 0.6.0) on Windows/AMD: ~2.4 GB of freshly loaded MOSSImage weights occasionally land in shared GPU memory despite ~6 GB free VRAM, sampling becomes ~5x slower (random, ~12% of runs)

Open
  • Is something wrong?
#17,656 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
python, pytorch
Domain
backend, performance

Research direction

The report points to ComfyUI's dynamic VRAM loading path and comfy-aimdo's WDDM handling, but names no source files or tests. Start by tracing where ModelPatcherDynamic loads MOSSImage weights and how aimdo reports resident versus shared memory; compare the supplied slow and normal run measurements. Done means identifying why about 2.4 GB is intermittently placed in shared memory and documenting a reproducible diagnosis or fix.

Written by the indexing model from the issue text.

Description

Environment

  • ComfyUI 0.43.0, Windows 10 Pro (10.0.19045), Python 3.12.9
  • PyTorch 2.14.0+rocm10.1.0, ROCm version (7, 16), AMD Radeon RX 6900 XT (gfx1030), 16368 MB VRAM, 32 GB system RAM, AMD driver 32.0.21045.16010
  • comfy-aimdo 0.6.0 (latest on PyPI), aimdo log: shmem-detect.c ... WDDM adapter match: AMD Radeon RX 6900 XT, cuda-detour.c: installing 6 hooks
  • Launch args: --listen (IP) --port (port) --disable-smart-memory --use-quad-cross-attention --fp16-unet --disable-pinned-memory --vram-headroom 2.5
  • Workflow: MOSS Image (moss_anime_turbo_sft_int8_convrot, ~5.2 GB, ModelPatcherDynamic) + Qwen3.5-9B text encoder (17 GB staged) + a separate llama-server process for prompt enhancement (sleeps between runs). Models are unloaded after every prompt.

Problem

Normally a run takes ~135-140 s (sampling 83 s). In about 1 of 8 runs the sampling step takes 430-480 s while all other phases are unchanged. Reproduced twice with full measurements today (11:35 and 13:15), and several more times earlier.

What I measured (GPU memory counters once per second, Windows "GPU Adapter Memory" dedicated / shared, plus per process)

Right after "Requested to load MOSSImage" (text encoder gets unloaded, MOSSImage prepared for dynamic VRAM):

normal runs slow runs (11:35 / 13:15)
dedicated VRAM 1-3 s after load ~12.2 GB 9.95 GB / 10.0 GB
shared GPU memory ~0.46 GB 2.69 GB / 2.61 GB
dedicated + shared ~12.6 GB ~12.6 GB (same)
get_total_vram_usage() (aimdo) 12.45-12.69 GB 11.56 GB
ModelPatcherDynamic loaded_mb of 5163 3137-3361 2241
  • The same amount is allocated, but ~2.4 GB ends up in shared (system RAM backed) memory instead of dedicated VRAM, although ~6 GB of VRAM was free at that moment. Dedicated stays pinned near 10 GB in both slow runs, even though the state before the load differed (11.3 GB resp. 15.4 GB).
  • It is decided within about 1-2 s after the text encoder is unloaded and MOSSImage is loaded. It never changes during sampling afterwards, which then runs ~5x slower (weights are streamed from system memory every step).
  • Torch allocator state before the load is clean in slow and normal runs (reserved 2 MB, no alloc retries). Other GPU processes unchanged (browser, Discord constant). RAM free at load was comparable (7-12 GB) in slow and normal runs.
  • --vram-headroom 3.5 (instead of 2.5) made no difference (slow run happened anyway).
  • Same pattern in the separate llama-server process (9.5 GB GGUF loaded with -ngl auto --fit on): in some runs ~2.4 GB lands in shared memory instead of VRAM (LLM phase 30 s -> 55-90 s), so this may be a driver/WDDM-level demotion rather than something specific to MOSS.
  • With --disable-dynamic-vram the machine ran out of virtual memory (Windows error 1455, whole system froze), so that is not a usable workaround here.

Question

Is there a known race / budget issue on WDDM (e.g. VRAM from the just-unloaded text encoder not yet returned when the new vbar maps its pages, or a WDDM per-process budget decision) that could produce this? Is there a way to log what aimdo decides for the vbar at load time (resident vs. host pages)? I can provide more logs, the per-second GPU traces and per-run summaries.

lauf-1135-summary.json
lauf-1315-log-slice.txt
lauf-1315-summary.json
lauf-1135-log-slice.txt

Dominant language
Python
Stars
146k
Forks
17.4k
Avg merge
19h 44m
Merged PRs (30d)
178

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Comfy-Org/ComfyUI

All issues in Comfy-Org/ComfyUI

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.