vllm-project/vllm
Auf GitHub ansehen[Feature]: Default eplb num_redundant_experts to the lowest valid value if unspecified
Open
#30.075 geöffnet am 4. Dez. 2025
feature requestgood first issuehelp wantedunstale
Repository-Metriken
- Stars
- (80.034 Stars)
- PR-Merge-Metriken
- (Durchschn. Merge 3T 17h) (993 gemergte PRs in 30 T)
Beschreibung
🚀 The feature, motivation and pitch
EPLB requires the number of experts to be chosen up front and there is a known minimum valid value that can be derived from the vllm startup configuration. Since extra EPLB experts trades kv cache memory for potential performance improvements, but that is not guaranteed to pay off, having the EPLB value default to the minimum valid value would reduce friction on enabling EPLB the first time until users are ready to tune.
As a consequence, it would also streamline templating the same config to work across multiple EP sizes for the default case.
Alternatives
No response
Additional context
No response
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.