Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Darwin Go backends are stamped "SDK 12.0", so Metal compiles ggml kernels at an old language version (bf16 kernels missing → SIGSEGV; M5 tensor API disabled)

Chiusa
  • Qualcosa non va?
#12,266 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
55/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
github-actions, go, macos
Ambito
backend, build-system, ci-cd

Direzione di ricerca

Start with .github/workflows/backend.yml, go.mod, and the referenced Darwin backend Makefiles, especially stablediffusion-ggml. Inspect the Go version and linker settings used by the Darwin job, then build a backend and verify its minos and sdk values with vtool. Done means the produced hosts carry a modern SDK stamp and the affected Metal workloads no longer omit bf16 or tensor support.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug unconfirmed

LocalAI version:
v4.10.0 (7ad0cbf259f0c7bf9920fe2438fc3630ecd6c672). Backend images:

  • metal-stablediffusion-ggml sha256:5cf7b37bd3f28f890acc815dab20b77477d83c3f2157d154d7847f2976976002
  • metal-stablediffusion-ggml-development sha256:346eb626b37d6278a362d88c8bf4a7145901e716bcf2414de1ff608325bd4bcd
  • metal-acestep-cpp sha256:4e4423e393b781b43fda23ba879faf8dbd07bcac277a95f8be58f0a2de034d53

Environment, CPU architecture, OS, and Version:
Mac Studio, Apple M5 Ultra, 256 GB, macOS 27.0, bare metal.

Darwin studio 27.0.0 Darwin Kernel Version 27.0.0: Tue Aug 11 21:06:40 PDT 2026; root:xnu-13432.1.9~1/RELEASE_ARM64_T6050 arm64

Describe the bug
ggml's Metal backend compiles its kernels from embedded source at runtime without setting a language version. The bf16 kernels only exist at Metal ≥ 3.1 (e.g.), but ggml decides bf16 support from the GPU family alone. When Metal compiles at an older version, the bf16 kernels are silently left out while ggml still dispatches bf16 ops to them: Function kernel_mul_mm_bf16_f32 was not found in the library, then SIGSEGV. The same default makes the M5 tensor-API self-test fail and disable itself, because it needs Metal 4.0 (see ggml-org/llama.cpp#27461).

Metal picks that default from the SDK version stamped in the main executable (LC_BUILD_VERSION), not from the library doing the compiling. The ggml library in these backends is built with a current SDK, but it is loaded via purego into a Go host built with CGO_ENABLED=0. Go ≤ 1.26's internal linker hard-codes that stamp to macOS 12.0 / SDK 12.0. Metal therefore treats the backend as a macOS 12 app, and compiles at the macOS 12 language level. No code is functionally wrong; only the stamp is. Changing just the stamp fixes it (see Additional context). The same symptom from an old-SDK link: ggml-org/llama.cpp#21381.

To Reproduce

  1. Install metal-stablediffusion-ggml and a model with bf16 tensors, e.g. flux.2-klein-9b from the gallery.
  2. Generate an image. The backend dies (the UI shows rpc error … connection refused).
  3. Check the host's stamp: 
    vtool -show-build <backends>/metal-stablediffusion-ggml/stablediffusion-ggml | grep -E 'minos|sdk'
#     minos 12.0
#       sdk 12.0

Expected behavior
bf16 kernels compile and the image is generated. On M5, the tensor API passes its self-test.

Logs
--debug doesn't help here: LocalAI's log shows only

Backend process exited unexpectedly id="flux.2-klein-9b" … process="run.sh" exitCode="2" stderr="fault   0x0"

The real error is in the backend's stderr, captured by appending 2>>file to the exec line in run.sh (cf. #11529):

ggml_metal_device_init: testing tensor API for f16 support
ggml_metal_library_init_from_source: error compiling source
ggml_metal_device_init: - the tensor API is not supported in this environment - disabling
ggml_metal_device_init: has bfloat            = true
ggml_metal_device_init: has tensor            = false
…
ggml_metal_library_compile_pipeline: compiling pipeline: base = 'kernel_mul_mm_bf16_f32', name = 'kernel_mul_mm_bf16_f32_bci=0_bco=0_ne12=1_ne13=1_r2=1_r3=1'
ggml_metal_library_compile_pipeline: failed to compile pipeline: base = 'kernel_mul_mm_bf16_f32', name = 'kernel_mul_mm_bf16_f32_bci=0_bco=0_ne12=1_ne13=1_r2=1_r3=1'
ggml_metal_library_compile_pipeline: Error Domain=MTLLibraryErrorDomain Code=5 "Function kernel_mul_mm_bf16_f32 was not found in the library" UserInfo={NSLocalizedDescription=Function kernel_mul_mm_bf16_f32 was not found in the library}
SIGSEGV: segmentation violation

After re-stamping only the host (below), with the same library and model:

ggml_metal_device_init: testing tensor API for f16 support
ggml_metal_device_init: testing tensor API for bfloat support
ggml_metal_device_init: has bfloat            = true
ggml_metal_device_init: has tensor            = true

The image then generates correctly: 1024×1024, HTTP 200.

Additional context

Workaround: re-stamp the host binary. The host is the binary on the exec line of the backend's run.sh. Re-stamp the SDK (keeping minos) and ad-hoc re-sign it:

B=<backends>/metal-stablediffusion-ggml; P=stablediffusion-ggml
cp -p $B/$P $B/$P.orig
codesign --remove-signature $B/$P
vtool -set-build-version macos 12.0 26.0 -replace -output $B/$P.new $B/$P
codesign -s - $B/$P.new && mv $B/$P.new $B/$P

A backend reinstall undoes this. Verified on:

  • metal-stablediffusion-ggml (FLUX.2-klein-9B, FLUX.2-dev);
  • metal-stablediffusion-ggml-development (Qwen-Image 2.1);
  • metal-acestep-cpp (acestep-cpp-turbo-4b).

Every Go backend built with CGO_ENABLED=0 and Metal has the same stamp, so all are candidates:
acestep-cpp,
crispasr,
depth-anything-cpp,
face-detect,
locate-anything-cpp,
magpie-tts-cpp,
moss-transcribe-cpp,
moss-tts-cpp,
nemo-speech-cpp,
omnivoice-cpp,
parakeet-cpp,
qwen3-tts-cpp,
rfdetr-cpp,
sam3-cpp,
stablediffusion-ggml,
trellis2cpp,
vibevoice-cpp,
vllm-cpp,
voice-detect,
voxtral,
whisper.

Fix: build the hosts with a modern SDK stamp.

I have not personally confirmed either of these fixes on the build-side, but from applying the hack described I can confirm that this version fix resolves the crashes on models that require newer Metal versions.

Lingua principale
Go
Stelle
49.2k
Fork
4.5k
Merge medio
1g 7h
PR unite (30g)
357

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di mudler/LocalAI

Tutte le issue di mudler/LocalAI

Issue simili

Altre issue su Go

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.