E2E support for Qwen3 dense; fused QK-RMSNorm + RoPE kernel
Maintainers usually reply within 8 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- python
- Domain
- machine-learning, performance
Research direction
Start by comparing the existing Qwen2 implementation with Qwen3.5 and the Qwen3RMSNorm approach described in the issue. Then read _rope_kernel tiling and apply_tilegym_kernel_to_qwen3 to determine the scope of fused_qk_norm_rope; done means E2E support for Qwen3 dense models and a reusable GQA-aware fused operation as proposed.
Written by the indexing model from the issue text.
Description
Currently, Qwen3 dense (0.6B-32B) isn't implemented, despite support for Qwen2 and Qwen3.5. Implementation itself would match almost perfectly onto existing ops (similar to Qwen2), with an additional per-head QK-RMSNorm.
I plan to approach this in 2 PRs:
- Integrating Qwen3 using existing ops, with the same shape as the Qwen2 implementation (alongside a
Qwen3RMSNormcoveringq_normandk_norm). - Fuse q norm, k norm, and RoPE into one
fused_qk_norm_ropekernel that is GQA-aware. I'd implement it as a core op, following the existing_rope_kerneltiling and containing params to apply the general op to all supported models that can use it (Qwen3, Gemma3, Qwen3.5).
I'll be developing on my 3060ti with sm_86.
Some quick questions:
- Anything already being worked on internally for Qwen3 dense?
- Qwen3.5 already uses
apply_tilegym_kernel_to_qwen3. Should I name the new one with_qwen3_dense, or rename the old one? - Does fused kernel fit as a core op, since its reusable accross Gemma3/Qwen3.5, or should I keep it local to Qwen3?
- Dominant language
- Python
- Stars
- 830
- Forks
- 92
- Avg merge
- 8d 11h
- Merged PRs (30d)
- 8
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from NVIDIA/TileGym
-
Difficulty 5/5 Over a week Newbie friendliness 12/100
Maintainers usually reply within 8 days
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
NVIDIA/TileGym#195 · 2 comments ·
Maintainers usually reply within 8 days
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
NVIDIA/TileGym#191 · 1 comment ·
Maintainers usually reply within 8 days
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
NVIDIA/TileGym#190 · 2 comments ·
Maintainers usually reply within 8 days
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
NVIDIA/TileGym#169 · 5 comments · 1 reaction ·
Maintainers usually reply within 8 days
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
enhancement good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
elviskahoro/sdk-python-attio#144 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
RenyiELBO raises TypeError when model has no sample sitesPossibly taken @alibb007 claimed this today. Open
Difficulty 1/5 Under an hour Newbie friendliness 85/100
Maintainers usually reply within 1 day