CUDA single GPU: --gpu-vram + --ssd-streaming refused as "multi-GPU placement" (misleading message)
Nobody has claimed this yet.
Assessment
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Newbie friendliness
- 82/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- c
- Domain
- ai-infra-agents
Research direction
Reproduce the single-GPU command in the issue, then inspect engine_classify_multi_tier() and the guard at ds4.c:57748. Choose the message-only fix or the documented single-GPU behavior change, and verify the refusal no longer presents this case as multi-GPU placement.
Written by the indexing model from the issue text.
Description
Summary
On a single CUDA GPU, passing an explicit --gpu-vram N together with --ssd-streaming refuses to start with:
ds4: --ssd-streaming is not compatible with multi-GPU placement
There is no multi-GPU placement — one device, one budget. The message sent us down the wrong path twice before we found the working invocation (drop --gpu-vram, streaming manages VRAM itself).
Environment
- RunPod pod, 1× RTX 3090 24 GB (sm_86), driver 580.65.06, 251 GB RAM
make cuda CUDA_ARCH=sm_86, commitc1d4597- Model:
ds4f-q2(81 GB)
Repro
./ds4-server -m ./ds4flash.gguf --cuda --ssd-streaming --gpu-vram 21 --ctx 8192 # refused
./ds4-server -m ./ds4flash.gguf --cuda --ssd-streaming --ctx 8192 # works
Where it comes from (reading the source)
With an explicit budget, engine_classify_multi_tier() runs the placement: an 81 GB model against a 21 GB budget classifies most layers as CPU-spill entries, the placement becomes tiered, and e->multi_tier turns on with a single GPU. The guard at ds4.c:57748 then fires, but its message names multi-GPU rather than the real condition (explicit budget produced a tiered placement that streaming does not support).
Suggested fix (either is fine)
- Message: say what happened — e.g.
--ssd-streaming does not support tiered placement; with an explicit --gpu-vram budget smaller than the model, layers spill to CPU. Drop --gpu-vram to let streaming manage VRAM. - Or behavior: with
n_gpus == 1and streaming requested, ignore the explicit budget (with a log line) instead of refusing.
Happy to send a small PR for the message variant if useful.
- Dominant language
- C
- Stars
- 22.5k
- Forks
- 2.2k
- Avg merge
- 2d 13h
- Merged PRs (30d)
- 5
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from antirez/ds4
-
Difficulty 2/5 1-3 hours Newbie friendliness 92/100
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 1/5 Under an hour Newbie friendliness 85/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Similar issues
-
level/task module/gcp type/bug
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
-
docs
Difficulty 1/5 Under an hour Newbie friendliness 85/100
-
P3 sonic-vpp
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
sonic-net/sonic-buildimage#29662 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 94/100
spack/spack-packages#6586 ·
-
category:port-update
Difficulty 2/5 1-3 hours Newbie friendliness 72/100