[BUG] llama-perplexity crashes with CUDA illegal instruction on RTX 3090 (--kl-divergence, -fit on)
Personne n'a encore pris cette issue.
Évaluation
- Difficulté
- 4/5
- Temps estimé
- 3-5 jours
- Accessibilité débutants
- 25/100
Piste de recherche
The crash is raised in ggml_cuda_compute_forward at ggml/src/ggml-cuda/ggml-cuda.cu line 2547, during a MUL_MAT op, and only with -fit on. Start by reproducing with llama-perplexity --kl-divergence and the -fit on versus -fit off flags, then compare which tensors and devices differ between the two runs. Done means the default -fit on run completes all chunks with KLD statistics printed.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Description
Name and Version
- Version: 0.4.8 (build 12480, commit ce1a9994f)
- Built with: Clang 20.1.8 for Windows x86_64
- ggml op count: 105
- SHA256: 476353A6BB06D59578DC6E607466AC799AE74B03B1A9089A03B9912C0A185EF8
- Build date: 2026-10-14 23:34:54
Operating systems
Windows
GGML backends
CUDA
Hardware
System:
- OS: Windows 11 IoT Enterprise LTSC 10.0.26100
- CPU: AMD Ryzen 9 5950X (16C/32T)
- RAM: 64 GB
- Shell: PowerShell 5.1.26100.9711
GPU:
- Model: NVIDIA GeForce RTX 3090 (Ampere, compute capability 8.6)
- VRAM: 24576 MiB
- Driver: 617.42
- CUDA UMD Version: 13.4
Models
- Name: Qwen3.8-27B-MTP-IMAT-IQ4_XS-Q8nextn.gguf
- Source: https://huggingface.co/localweights/Qwen3.8-27B-MTP-IMAT-IQ4_XS-Q8nextn-GGUF/resolve/main/Qwen3.8-27B-MTP-IMAT-IQ4_XS-Q8nextn.gguf
- Quant: IQ4_XS (imatrix), with embedded NextN/MTP head (Q8_0)
- Architecture: Qwen 3.8 Dense 27B
- Note: this GGUF contains an embedded NextN/MTP head (Q8_0). llama-perplexity does not use MTP, but mentioning it in case the loader treats MTP-embedded files differently.
Problem description & steps to reproduce
Summary
llama-perplexity.exe crashes with CUDA error: an illegal instruction was encountered when computing KL divergence (--kl-divergence + --kl-divergence-base) on Qwen3.8-27B IQ4_XS with -fit on (default). The same command with -fit off completes all 72 chunks successfully. Mainline llama.cpp (ggml-org) runs the same command to completion without errors in both modes.
Reproduction
Crashing variant (-fit on, the default — no flag needed):
.\llama-perplexity.exe `
-m "<MODEL_DIR>\Qwen3.8-27B-MTP-IMAT-IQ4_XS-Q8nextn.gguf" `
-f "<MODEL_DIR>\wiki.test.raw" `
-fa on `
-ngl 99 `
-c 4096 `
--kl-divergence-base "<MODEL_DIR>\kld-baseline-fp16.kld" `
--kl-divergence `
--chunks 10 `
--seed 1337 `
--threads 1
Working variant (-fit off):
.\llama-perplexity.exe `
-m "<MODEL_DIR>\Qwen3.8-27B-MTP-IMAT-IQ4_XS-Q8nextn.gguf" `
-f "<MODEL_DIR>\wiki.test.raw" `
-fa on `
-ngl 99 `
-c 4096 `
-fit off `
--kl-divergence-base "<MODEL_DIR>\kld-baseline-fp16.kld" `
--kl-divergence `
--chunks 10 `
--seed 1337 `
--threads 1
Baseline was created the same way without --kl-divergence-base/--kl-divergence.
Expected behavior
llama-perplexity --kl-divergence should complete successfully in both modes (-fit on and -fit off), as it does on ggml-org/llama.cpp.
Preliminary investigation (as requested in the template)
I ran the requested llama-completion -fit off test and both llama-perplexity variants.
Test-fit on``-fit off``llama-completion✅ PASS✅ PASSllama-perplexity --kl-divergence❌ CRASH (chunk 16)✅ PASS (all 72 chunks)
Conclusion: the crash is tied to -fit on in combination with --kl-divergence. Base inference (llama-completion) works fine in both modes.
Additional context
The same model, same --kl-divergence-base file, same arguments run to completion on ggml-org/llama.cpp (no crash, KLD statistics printed normally).
The bug is specific to BeeLlama 0.4.8 with -fit on.
Workaround: use -fit off when running llama-perplexity --kl-divergence.
I have not been able to test KVarN cache types (-ctk kvarn4 -ctv kvarn4) at all, because llama-perplexity crashes before finishing — which is the main reason I'm reporting this.
First Bad Commit
No response
Relevant log output
Crash log with -fit on (default)
Last 30 lines before crash:
10 6.1698 ± 0.1068 0.00337 ± 0.00101 0.00560 ± 0.00124 1.894 ± 0.244 % 98.764 ± 0.077 %
11 6.3279 ± 0.1055 0.00360 ± 0.00093 0.00523 ± 0.00113 1.848 ± 0.228 % 98.779 ± 0.073 %
12 6.3467 ± 0.1008 0.00318 ± 0.00086 0.00485 ± 0.00104 1.780 ± 0.217 % 98.775 ± 0.070 %
13 6.4863 ± 0.1007 0.00352 ± 0.00083 0.00486 ± 0.00097 1.836 ± 0.214 % 98.730 ± 0.069 %
14 6.6796 ± 0.1006 0.00357 ± 0.00079 0.00456 ± 0.00090 1.781 ± 0.205 % 98.730 ± 0.066 %
15 6.6054 ± 0.0956 0.00336 ± 0.00074 0.00433 ± 0.00084 1.731 ± 0.197 % 98.733 ± 0.064 %
16 6.4418 ± 0.0896 0.00324 ± 0.00071 0.00424 ± 0.00080 1.777 ± 0.196 % 98.730 ± 0.062 %
1.07.274.910 E ggml_cuda_compute_forward: MUL_MAT failed
1.07.275.095 E CUDA error: an illegal instruction was encountered
1.07.275.100 E current device: 0, in function ggml_cuda_compute_forward at ggml/src/ggml-cuda/ggml-cuda.cu:2547
1.07.275.100 E err
ggml/src/ggml-cuda/ggml-cuda.cu:113: CUDA error
Working log with -fit off
With -fit off the same command completes all 72 chunks and prints the KLD statistics block. The log ends with:
====== KL divergence statistics ======
Mean KLD: 0.008461 ± 0.000782
99.9% KLD: 1.111757
99.0% KLD: 0.012812
95.0% KLD: 0.003030
Median KLD: 0.000358
What I would like to see
llama-perplexity --kl-divergence completes successfully with -fit on (the default), so KLD benchmarks work out of the box.
Ideally also with KVarN cache types (-ctk kvarn4 -ctv kvarn4), which I haven't been able to test yet because the crash happens before finishing.
- Langage dominant
- C++
- Étoiles
- 1.2k
- Forks
- 80
- Merge moyen
- 4 j 20 h
- PR mergées (30 j)
- 2
Préparer son environnement
Ce projet ne fournit ni conteneur de développement, ni Dockerfile, ni guide de contribution : l'installation est à votre charge. Commencez par son README, et consultez notre guide de la première contribution pour les étapes générales.
Par où commencer
- Lisez l'issue en entier, puis le guide de contribution du projet.
- Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
- Forkez le dépôt et travaillez sur une branche.
- Ouvrez une pull request qui référence le numéro de l'issue.
Autres issues de Anbeeld/beellama.cpp
-
[BUG] KVarN assertion failure in build_inp_mem_hybrid_iswa() due to uninitialized self_kvarn_rotOuverte
Difficulté 2/5 1-3 heures Accessibilité débutants 68/100
Anbeeld/beellama.cpp#205 ·
-
Difficulté 3/5 Une demi-journée Accessibilité débutants 58/100
Anbeeld/beellama.cpp#206 ·
-
Difficulté 3/5 1-2 jours Accessibilité débutants 57/100
Anbeeld/beellama.cpp#204 ·
-
Difficulté 4/5 3-5 jours Accessibilité débutants 45/100
Anbeeld/beellama.cpp#203 · 2 commentaires ·
-
Difficulté 4/5 3-5 jours Accessibilité débutants 45/100
Anbeeld/beellama.cpp#202 · 1 commentaire ·
Toutes les issues de Anbeeld/beellama.cpp
Issues similaires
-
Difficulté 2/5 1-3 heures Accessibilité débutants 72/100
CachyOS/packageinstaller#144 ·
Les mainteneurs répondent en général sous 1 jour
-
documentation
Difficulté 1/5 Moins d'une heure Accessibilité débutants 82/100
TomerAberbach/profiler-md#951 ·
Les mainteneurs répondent en général sous 1 jour
-
[Bug] Default key routing (JavaStringHash) picks a different partition than the Java client for about half of keysPeut-être pris @0z5a l’a pris aujourd’hui. Ouverte
Difficulté 2/5 1-3 heures Accessibilité débutants 82/100
apache/pulsar-client-go#1575 ·
Les mainteneurs répondent en général sous 2 jours
-
Difficulté 2/5 1-3 heures Accessibilité débutants 72/100
pocoproject/poco#5764 ·
Les mainteneurs répondent en général sous 2 jours
-
bug cpp-core-guidelines low-priority
Difficulté 2/5 1-3 heures Accessibilité débutants 62/100
NatronGitHub/openfx-supportext#164 ·
Les mainteneurs répondent en général sous 1 jour