vllm-project/vllm
Auf GitHub ansehen[Feature]: Priority scheduling supports preemption of requests in the running queue by requests in the waiting queue
Open
#40.004 geöffnet am 16. Apr. 2026
feature requesthelp wanted
Repository-Metriken
- Stars
- (80.034 Stars)
- PR-Merge-Metriken
- (Durchschn. Merge 9T 2h) (921 gemergte PRs in 30 T)
Beschreibung
🚀 The feature, motivation and pitch
The current priority scheduling only supports evicting low-priority requests from the running queue when resources are insufficient. However, when scheduling the waiting queue, if pending requests cannot be scheduled (for example, when the number of requests in the running queue has reached max_num_seqs), even high-priority requests cannot preempt requests in the running queue.
Alternatives
No response
Additional context
No response
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.