vllm-project/vllm
[Feature]: Priority scheduling supports preemption of requests in the running queue by requests in the waiting queue
開放
#40,004 建立於 2026年4月16日
feature requesthelp wanted
倉庫指標
- 星標
- (80,034 顆星)
- PR 合併指標
- (PR 指標待抓取)
描述
🚀 The feature, motivation and pitch
The current priority scheduling only supports evicting low-priority requests from the running queue when resources are insufficient. However, when scheduling the waiting queue, if pending requests cannot be scheduled (for example, when the number of requests in the running queue has reached max_num_seqs), even high-priority requests cannot preempt requests in the running queue.
Alternatives
No response
Additional context
No response
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.