vllm-project/vllm

[Feature]: Priority scheduling supports preemption of requests in the running queue by requests in the waiting queue

Open

#40004 opened on Apr 16, 2026

View on GitHub
 (5 comments) (0 reactions) (0 assignees)Python (80,034 stars) (16,816 forks)batch import
feature requesthelp wanted

Description

🚀 The feature, motivation and pitch

The current priority scheduling only supports evicting low-priority requests from the running queue when resources are insufficient. However, when scheduling the waiting queue, if pending requests cannot be scheduled (for example, when the number of requests in the running queue has reached max_num_seqs), even high-priority requests cannot preempt requests in the running queue.

Alternatives

No response

Additional context

No response

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

Contributor guide

Tech stack
python
Domain
backendperformanceai
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
4
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
mostly clear
Prerequisites
Understanding of vLLM schedulerPython programmingConcurrency concepts
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
25
Research direction
The issue involves modifying the priority scheduling logic in the vLLM scheduler. Key files to examine include vllm/v1/core/scheduler.py and vllm/v1/core/schedule.py. The current implementation only preempts low priority requests from the running queue when resources are insufficient. To allow high priority waiting requests to preempt running requests, you'll need to add logic to check priority when scheduling the waiting queue and potentially evict lower priority running requests. Review the existing preemption mechanism and the scheduling loop. Start by understanding the data structures for request queues and priority handling.