Windows: CUDA Backend Fails to Initialize RTX 4050 While Vulkan Backend Works
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 38/100
Research direction
Start by reproducing the Windows CUDA failure with the reported nemo-speech --verbose transcribe command and inspect the ggml_cuda_init initialization path mentioned in the output. Compare the CUDA failure with the working Vulkan run on the same machine; done means the CUDA initialization issue is diagnosed or the Windows build provides useful additional diagnostics.
Written by the indexing model from the issue text.
Description
Version
• Build v0.3.0 (20999cc)
• Tested package: nemo-speech-0.3.0-windows-x86_64-vulkan.zip
• Tested package: nemo-speech-0.3.0-windows-x86_64-cuda.zip
Environment
• OS: Windows 11
• GPU: NVIDIA RTX 4050 Laptop GPU
• NVIDIA Driver: 555.97
• CUDA Version: 12.5
NVIDIA GPU Detection
nvidia-smi -L
GPU 0: NVIDIA GeForce RTX 4050 Laptop GPU
(UUID: GPU-1222eb33-44c3-5916-94dd-37982642f7fa)
Results
• ✅ Vulkan backend initializes successfully
• ✅ Vulkan transcription works
• ✅ Vulkan diarization works
• ✅ AMD Radeon 780M processes inference
• ✅ RTX 4050 is detected by nvidia-smi
• ❌ CUDA backend fails to initialize/run
• ❌ ggml_cuda_init returns (null)
Notes
The Vulkan backend works correctly using the same:
• Model
• Audio file
• Machine
• Build
This suggests the issue is specific to CUDA initialization or device detection rather than the model, audio file, or transcription pipeline.
Vulkan Output
nemo-speech --verbose transcribe audio.wav
--format srt
--model parakeet-tdt-0.6b-v3.q8_0.gguf
--output output.srt
[nemo-speech] transcribe session started
transcribe: model=parakeet-tdt-0.6b-v3.q8_0.gguf inputs=1 concurrency=1 device=0
[asr] model=parakeet-tdt-0.6b-v3.oss-align.q8_0 head=tdt backend=Vulkan0
[asr] mode=offline head=tdt attention=full-context
CUDA Error
transcribe: model=parakeet-tdt-0.6b-v3.q8_0.gguf inputs=1 concurrency=1 device=0
ggml_cuda_init: failed to initialize CUDA: (null)
nemo-speech transcribe: use_gpu=true but no matching GPU device found (gpu_device_idx=0)
[nemo-speech] transcribe session failed (exit code 1)
Expected Behavior
CUDA backend should initialize and run successfully on the RTX 4050, similar to how the Vulkan backend initializes and runs successfully on the AMD Radeon 780M.
Additional Observation
The --verbose flag produces some additional output, such as model and backend information for the Vulkan backend.
According to --help:
--verbose Emit additional diagnostics on stderr
However, during CUDA initialization the diagnostic output remains very limited.
The CUDA backend reports only:
ggml_cuda_init: failed to initialize CUDA: (null)
nemo-speech transcribe: use_gpu=true but no matching GPU device found (gpu_device_idx=0)
[nemo-speech] transcribe session failed (exit code 1)
No further CUDA initialization details or error information are emitted, making it difficult to diagnose the failure.
The NVIDIA GPU is visible to CUDA tooling via nvidia-smi, but the CUDA backend is unable to detect or initialize it.
This may be relevant on hybrid AMD/NVIDIA systems where Vulkan selects the integrated AMD Radeon 780M while CUDA targets the discrete NVIDIA RTX 4050.
Is there a way to enable more detailed CUDA logging or debug telemetry in the Windows builds to help diagnose CUDA initialization failures?
Summary
Vulkan (AMD Radeon 780M): PASS
CUDA (NVIDIA RTX 4050): FAIL
This appears to be a CUDA initialization and/or device detection issue specific to the Windows CUDA build.
- Dominant language
- C++
- Stars
- 207
- Forks
- 47
- Avg merge
- 13h 10m
- Merged PRs (30d)
- 12
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from NVIDIA/NeMo-Speech.cpp
-
Difficulty 3/5 Half a day Newbie friendliness 35/100
NVIDIA/NeMo-Speech.cpp#101 ·
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 45/100
NVIDIA/NeMo-Speech.cpp#98 ·
Maintainers usually reply within 1 day
-
Difficulty 3/5 1-2 days Newbie friendliness 55/100
NVIDIA/NeMo-Speech.cpp#94 ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
NVIDIA/NeMo-Speech.cpp#93 ·
Maintainers usually reply within 1 day
-
Difficulty 3/5 1-2 days Newbie friendliness 57/100
NVIDIA/NeMo-Speech.cpp#92 ·
Maintainers usually reply within 1 day
All issues in NVIDIA/NeMo-Speech.cpp
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
FujiNetWIFI/fujinet-firmware#2261 ·
Maintainers usually reply within 1 day
-
bug difficulty: easy good first issue help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
nagisa-kamishiro/rosehip#73 ·
Maintainers usually reply within 1 day
-
mode_has_nan gives MSVC warning C4702 (unreachable code)Possibly taken @ibmibmibm claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
pocoproject/poco#5626 ·
Maintainers usually reply within 2 days