Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Darwin Go backends are stamped "SDK 12.0", so Metal compiles ggml kernels at an old language version (bf16 kernels missing → SIGSEGV; M5 tensor API disabled)

Cerrado
  • ¿Algo no está bien?
#12,266 2 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 3 días

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
55/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
github-actions, go, macos
Área
backend, build-system, ci-cd

Línea de trabajo

Start with .github/workflows/backend.yml, go.mod, and the referenced Darwin backend Makefiles, especially stablediffusion-ggml. Inspect the Go version and linker settings used by the Darwin job, then build a backend and verify its minos and sdk values with vtool. Done means the produced hosts carry a modern SDK stamp and the affected Metal workloads no longer omit bf16 or tensor support.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

bug unconfirmed

LocalAI version:
v4.10.0 (7ad0cbf259f0c7bf9920fe2438fc3630ecd6c672). Backend images:

  • metal-stablediffusion-ggml sha256:5cf7b37bd3f28f890acc815dab20b77477d83c3f2157d154d7847f2976976002
  • metal-stablediffusion-ggml-development sha256:346eb626b37d6278a362d88c8bf4a7145901e716bcf2414de1ff608325bd4bcd
  • metal-acestep-cpp sha256:4e4423e393b781b43fda23ba879faf8dbd07bcac277a95f8be58f0a2de034d53

Environment, CPU architecture, OS, and Version:
Mac Studio, Apple M5 Ultra, 256 GB, macOS 27.0, bare metal.

Darwin studio 27.0.0 Darwin Kernel Version 27.0.0: Tue Aug 11 21:06:40 PDT 2026; root:xnu-13432.1.9~1/RELEASE_ARM64_T6050 arm64

Describe the bug
ggml's Metal backend compiles its kernels from embedded source at runtime without setting a language version. The bf16 kernels only exist at Metal ≥ 3.1 (e.g.), but ggml decides bf16 support from the GPU family alone. When Metal compiles at an older version, the bf16 kernels are silently left out while ggml still dispatches bf16 ops to them: Function kernel_mul_mm_bf16_f32 was not found in the library, then SIGSEGV. The same default makes the M5 tensor-API self-test fail and disable itself, because it needs Metal 4.0 (see ggml-org/llama.cpp#27461).

Metal picks that default from the SDK version stamped in the main executable (LC_BUILD_VERSION), not from the library doing the compiling. The ggml library in these backends is built with a current SDK, but it is loaded via purego into a Go host built with CGO_ENABLED=0. Go ≤ 1.26's internal linker hard-codes that stamp to macOS 12.0 / SDK 12.0. Metal therefore treats the backend as a macOS 12 app, and compiles at the macOS 12 language level. No code is functionally wrong; only the stamp is. Changing just the stamp fixes it (see Additional context). The same symptom from an old-SDK link: ggml-org/llama.cpp#21381.

To Reproduce

  1. Install metal-stablediffusion-ggml and a model with bf16 tensors, e.g. flux.2-klein-9b from the gallery.
  2. Generate an image. The backend dies (the UI shows rpc error … connection refused).
  3. Check the host's stamp: 
    vtool -show-build <backends>/metal-stablediffusion-ggml/stablediffusion-ggml | grep -E 'minos|sdk'
#     minos 12.0
#       sdk 12.0

Expected behavior
bf16 kernels compile and the image is generated. On M5, the tensor API passes its self-test.

Logs
--debug doesn't help here: LocalAI's log shows only

Backend process exited unexpectedly id="flux.2-klein-9b" … process="run.sh" exitCode="2" stderr="fault   0x0"

The real error is in the backend's stderr, captured by appending 2>>file to the exec line in run.sh (cf. #11529):

ggml_metal_device_init: testing tensor API for f16 support
ggml_metal_library_init_from_source: error compiling source
ggml_metal_device_init: - the tensor API is not supported in this environment - disabling
ggml_metal_device_init: has bfloat            = true
ggml_metal_device_init: has tensor            = false
…
ggml_metal_library_compile_pipeline: compiling pipeline: base = 'kernel_mul_mm_bf16_f32', name = 'kernel_mul_mm_bf16_f32_bci=0_bco=0_ne12=1_ne13=1_r2=1_r3=1'
ggml_metal_library_compile_pipeline: failed to compile pipeline: base = 'kernel_mul_mm_bf16_f32', name = 'kernel_mul_mm_bf16_f32_bci=0_bco=0_ne12=1_ne13=1_r2=1_r3=1'
ggml_metal_library_compile_pipeline: Error Domain=MTLLibraryErrorDomain Code=5 "Function kernel_mul_mm_bf16_f32 was not found in the library" UserInfo={NSLocalizedDescription=Function kernel_mul_mm_bf16_f32 was not found in the library}
SIGSEGV: segmentation violation

After re-stamping only the host (below), with the same library and model:

ggml_metal_device_init: testing tensor API for f16 support
ggml_metal_device_init: testing tensor API for bfloat support
ggml_metal_device_init: has bfloat            = true
ggml_metal_device_init: has tensor            = true

The image then generates correctly: 1024×1024, HTTP 200.

Additional context

Workaround: re-stamp the host binary. The host is the binary on the exec line of the backend's run.sh. Re-stamp the SDK (keeping minos) and ad-hoc re-sign it:

B=<backends>/metal-stablediffusion-ggml; P=stablediffusion-ggml
cp -p $B/$P $B/$P.orig
codesign --remove-signature $B/$P
vtool -set-build-version macos 12.0 26.0 -replace -output $B/$P.new $B/$P
codesign -s - $B/$P.new && mv $B/$P.new $B/$P

A backend reinstall undoes this. Verified on:

  • metal-stablediffusion-ggml (FLUX.2-klein-9B, FLUX.2-dev);
  • metal-stablediffusion-ggml-development (Qwen-Image 2.1);
  • metal-acestep-cpp (acestep-cpp-turbo-4b).

Every Go backend built with CGO_ENABLED=0 and Metal has the same stamp, so all are candidates:
acestep-cpp,
crispasr,
depth-anything-cpp,
face-detect,
locate-anything-cpp,
magpie-tts-cpp,
moss-transcribe-cpp,
moss-tts-cpp,
nemo-speech-cpp,
omnivoice-cpp,
parakeet-cpp,
qwen3-tts-cpp,
rfdetr-cpp,
sam3-cpp,
stablediffusion-ggml,
trellis2cpp,
vibevoice-cpp,
vllm-cpp,
voice-detect,
voxtral,
whisper.

Fix: build the hosts with a modern SDK stamp.

I have not personally confirmed either of these fixes on the build-side, but from applying the hack described I can confirm that this version fix resolves the crashes on models that require newer Metal versions.

Lenguaje dominante
Go
Estrellas
49.2k
Forks
4.5k
Merge medio
1 d 7 h
PR fusionados (30 d)
340

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de mudler/LocalAI

Todos los issues de mudler/LocalAI

Issues similares

Más issues de Go

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.