good first issue
Metriche repository
- Star
- (28.442 star)
- Metriche merge PR
- (Merge medio 2g 1h) (1000 PR mergiate in 30 g)
Descrizione
Checklist
- 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
- 2. Please use English, otherwise it will be closed.
Motivation
Currently, EPLB is not supported in draft models, which constrains the parallelism size. For example, with EPLB, we can set the EP size as 72/144 for the DeepSeek model as it has 288 experts in total. However, these parallelism settings cannot be adopted when MTP is enabled as the draft model as has 256 experts.
Related resources
No response