vllm-project/vllm
[Feature]: Integrate fused `kMoEFinalizeARResidualRMSNorm` from FlashInfer
開放
#40,544 建立於 2026年4月21日
feature requesthelp wanted
倉庫指標
- 星標
- (80,034 顆星)
- PR 合併指標
- (平均合併 3天 17小時) (30 天內合併 993 個 PR)
描述
🚀 The feature, motivation and pitch
Available today via FlashInfer 0.6.8, we can leverage this fused MoE Finalize + ResidualAdd + AllReduce + RMSNorm https://github.com/flashinfer-ai/flashinfer/pull/2982.
This should be added as a new torch.compile custom pass. I think moe_finalize might still be inside the wrapped fused_moe op so this might require pulling that out.
Alternatives
No response
Additional context
No response
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.