Add support for blackwell and Cuda 13.3 for the cuda build

Open
#3,296 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
48/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Domain
build-system, performance

Research direction

Start by reproducing the shown Windows executable command with the CUDA backend on the NVIDIA GeForce RTX 5060 Ti, then inspect the repository's CUDA build entry points. Done means the CUDA build supports Blackwell and CUDA 13.3 and the reported inference failure no longer occurs.

Written by the indexing model from the issue text.

Description

backend:llamacpp bug needs feedback

Use case

Fix the issue with:
https://github.com/modelscope/FunASR/issues/3289

Implemented the fix for:
Cuda implementation broken

C:\Users\USER\Downloads\funasr-llamacpp-windows-x64-cuda>llama-funasr-sensevoice.exe -m sensevoice-small-q8.gguf -a sample.wav --backend cuda
ggml_cuda_init: found 1 CUDA devices (Total VRAM: 16310 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 16310 MiB

C:\Users\USER\Downloads\funasr-llamacpp-windows-x64-cuda>dir sample.wav
 Volume in drive C has no label.
 Volume Serial Number is ---

 Directory of C:\Users\USER\Downloads\funasr-llamacpp-windows-x64-cuda

07/19/2026  05:15 AM         1,077,326 sample.wav
               1 File(s)      1,077,326 bytes
               0 Dir(s)  89,731,526,656 bytes free

C:\Users\USER\Downloads\funasr-llamacpp-windows-x64-cuda>

C:\Users\USER\Downloads\funasr-llamacpp-windows-x64-cuda>llama-funasr-sensevoice.exe -m sensevoice-small-q8.gguf -a sample.wav
I'll record myself in English, no, you don't want.
[sensevoice] done 1.55s

Proposed solution

Add support for this card and cuda version!

Alternatives or workarounds

None

Expected impact

Improve performance for cuda

Dominant language
Python
Stars
20.4k
Forks
2k
Avg merge
4h 55m
Merged PRs (30d)
169

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from modelscope/FunASR

All issues in modelscope/FunASR

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.