[Model request]: Meta Muse Glimmer 30B

Start by comparing FastFlowLM's existing Gemma4e implementation with the linked Transformers and llama.cpp Muse Glimmer implementations. Scope the first milestone to text-only Q4/Q4NX support, accounting for the model's attention, normalization, RoPE/NoPE, and gating differences. Done means text-only Muse Glimmer inference works on the target Ryzen AI NPU; vision and DFlash can remain later stages.

Suggestion Description

Hi FastFlowLM team,

Could you consider adding NPU support for Meta's Muse Glimmer 30B?

Model links:

• Hugging Face collection: https://huggingface.co/collections/meta-models/muse-glimmer
• Base model: https://huggingface.co/meta-models/Muse-Glimmer-30B
• Official GGUF release: https://huggingface.co/meta-models/Muse-Glimmer-30B-GGUF
• Transformers implementation: https://github.com/huggingface/transformers/blob/main/src/transformers/models/muse_glimmer/modeling_muse_glimmer.py
• llama.cpp implementation: https://github.com/ggml-org/llama.cpp/pull/26841

Muse Glimmer is a dense 30B multimodal model designed for local agentic workflows. Its official approximately 4-bit quantizations keep the language model below 20 GB, which makes it particularly interesting for Ryzen AI Max / Strix Halo systems with 32 GB or more of unified memory.

Some relevant architecture details:

• 52 transformer layers, hidden size 6656
• [Local, Local, Local, Global] attention pattern
• 2048-token sliding window
• 32 query heads and 2 KV heads
• Q/K RMSNorm and gated attention output
• RoPE on local layers and NoPE on global layers
• 131K context length
• optional 50-layer ViT-G/14 perception encoder
• optional DFlash speculative drafter

There appears to be some architectural overlap with the existing Gemma4e implementation in FastFlowLM, especially around sliding/global attention, Q/K normalization, post-attention/post-FFN norms, and logit softcapping. However, Muse Glimmer also requires a separate attention gate, different dimensions, per-layer RoPE/NoPE behavior, and its own vision pipeline.

A staged implementation would already be very useful:

1. Text-only Q4/Q4NX support
2. Vision/perception encoder support
3. DFlash speculative decoding as an optional later feature

Even text-only support without vision or DFlash would make Muse Glimmer valuable for local coding, tool use, and agentic workloads on Ryzen AI NPUs.

Could you please clarify:

• Is Muse Glimmer already on the FastFlowLM roadmap?
• Could the existing Gemma4e NPU kernels be extended or reused for its text architecture?
• Would a new model-specific engine and .xclbin package be required?
• If community contributions are welcome, is there documentation or a contribution path for building the model engine and NPU kernel package?

Thank you for considering it, and for making Ryzen AI NPU inference accessible through FastFlowLM!

Operating System

No response

GPU

No response

ROCm Component

No response