vllm-project/vllm
GitHub ã§èŠã[Feature]: Implement Concurrent Partial Prefills In V1 Engine
Open
#14,003 opened on 2025幎2æ28æ¥
feature requesthelp wantedunstale
Repository metrics
- Stars
- Â (80,034 stars)
- PR merge metrics
-  (å¹³åããŒãž 3d 17h) (30d 㧠993 merged PRs)
説æ
ð The feature, motivation and pitch
In V0, we support concurrent partial prefills to avoid TTFT latency with long requests. Implement it in V1
cc @WoosukKwon
Alternatives
No response
Additional context
No response
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.