Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Misc. bug: Metal aborts on MUL_MAT with BF16 src1 at K >= 64, batch > 8 (no BF16 mul_mm kernel)

Open Beginner friendly
  • Is something wrong?
#30,973 4 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@rashid5925 is already working on this.

Since Oct 24, 2026.

  • #31105 by @rashid5925 — open

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
74/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
cpp
Domain
backend, testing

Research direction

Start with ggml_metal_op_mul_mat_use_mm in ggml/src/ggml-metal/ggml-metal-common.cpp and the test_mul_mat cases in tests/test-backend-ops.cpp. Check how BF16 src1 selects the mul_mm path and reproduce with the provided test-backend-ops command. Done when the BF16 cases avoid the missing Metal pipeline and the added tests pass on Metal.

Written by the indexing model from the issue text.

Description

bug-unconfirmed
Name and Version

master @ 7170cad (after b11813), built with AppleClang 21.0.0 for Darwin arm64

Operating systems

Mac

Which llama.cpp modules do you know to be affected?

Other (Please specify in the next section)

Command line
test-backend-ops test -b MTL0 -o MUL_MAT -p "type_a=bf16,type_b=bf16,m=64,n=17,k=64,"
Problem description & steps to reproduce

On Metal, a MUL_MAT whose src1 is BF16 aborts once K >= 64 and the batch (n) is above 8:

kernel not found in any metal library: base = 'kernel_mul_mm_bf16_bf16'
ggml_metal_encoder_set_pipeline: nil Metal pipeline (missing kernel; see compile_pipeline log above)

Cause:

  • ggml_metal_op_mul_mat_use_mm (ggml/src/ggml-metal/ggml-metal-common.cpp) chooses mul_mm from the shape alone.
  • mul_mm is only instantiated for F32 and F16 src1 (kernel_mul_mm_<src0>_f32 / _f16), so a BF16 src1 gets a nil pipeline.
  • mul_mv does have BF16 src1 kernels (kernel_mul_mv_bf16_bf16, kernel_mul_mv_f32_bf16), so smaller shapes work.

Affected pairs:

src0 x src1 K >= 64, n > 8
BF16 x BF16 abort (kernel_mul_mm_bf16_bf16 not found)
F32 x BF16 abort (kernel_mul_mm_f32_bf16 not found)
F16 x BF16, Q4_0 x BF16 not affected: Metal reports them unsupported, so they fall back to CPU

Why CI doesn't catch it: every existing BF16 src1 case in test-backend-ops uses K < 64 or n <= 8, so none reaches mul_mm.

Reach: no model graph in src/ produces a BF16 src1 for MUL_MAT. Even with a BF16 KV cache and flash attention off, the cache is src0 and the activations stay F32. It is reachable through the ggml API.

Steps to reproduce: stock master has no test that hits this. Add these two lines to make_test_cases_eval() in tests/test-backend-ops.cpp:

test_cases.emplace_back(new test_mul_mat(GGML_TYPE_BF16, GGML_TYPE_BF16, 64, 17, 64, {1, 1}, {1, 1}));
test_cases.emplace_back(new test_mul_mat(GGML_TYPE_F32,  GGML_TYPE_BF16, 64, 17, 64, {1, 1}, {1, 1}));

Then build and run the command above. It exits with code 134 (abort). The same happens at K = 256.

Hardware: Apple M4 (10-core GPU), macOS 27.0.1.

Proposed fix: keep BF16 src1 off the mul_mm path, so it uses the existing mul_mv kernels. This is a one-line condition in ggml_metal_op_mul_mat_use_mm

First Bad Commit

Unknown

Relevant log output
Logs 
ggml_metal_library_compile_pipeline: compiling pipeline: base = 'kernel_mul_mm_bf16_bf16', name = 'kernel_mul_mm_bf16_bf16_bci=0_bco=1_ne12=1_ne13=1_r2=1_r3=1'
ggml_metal_library_compile_pipeline: kernel not found in any metal library: base = 'kernel_mul_mm_bf16_bf16', name = 'kernel_mul_mm_bf16_bf16_bci=0_bco=1_ne12=1_ne13=1_r2=1_r3=1'
Testing 3 devices
Backend 1/3: MTL0
  Device description: Apple M4
<llama.cpp>/ggml/src/ggml-metal/ggml-metal-device.m:869: ggml_metal_encoder_set_pipeline: nil Metal pipeline (missing kernel; see compile_pipeline log above)
0   libggml-base.0.27.0.dylib    ggml_print_backtrace + 276
1   libggml-base.0.27.0.dylib    ggml_abort + 156
2   libggml-metal.0.27.0.dylib   ggml_metal_encoder_set_bytes + 0
3   libggml-metal.0.27.0.dylib   ggml_metal_op_mul_mat + 2208
4   libggml-metal.0.27.0.dylib   ggml_metal_op_encode + 2352
5   libggml-metal.0.27.0.dylib   __ggml_metal_set_n_cb_block_invoke + 180
6   libggml-metal.0.27.0.dylib   ggml_metal_graph_compute + 636
7   libggml-base.0.27.0.dylib    ggml_backend_compare_graph_backend + 492
8   test-backend-ops             test_case::eval(...) + 2408
Dominant language
C++
Stars
132k
Forks
24.6k
Avg merge
2d 1h
Merged PRs (30d)
500

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ggml-org/llama.cpp

All issues in ggml-org/llama.cpp

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.