good first issue
Repository-Metriken
- Stars
- (28.442 Stars)
- PR-Merge-Metriken
- (Durchschn. Merge 2T 1h) (1.000 gemergte PRs in 30 T)
Beschreibung
Checklist
- 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
- 2. Please use English, otherwise it will be closed.
Motivation
Currently, EPLB is not supported in draft models, which constrains the parallelism size. For example, with EPLB, we can set the EP size as 72/144 for the DeepSeek model as it has 288 experts in total. However, these parallelism settings cannot be adopted when MTP is enabled as the draft model as has 256 experts.
Related resources
No response