vllm-project/vllm
在 GitHub 查看[Feature]: Implement Concurrent Partial Prefills In V1 Engine
Open
#14,003 建立於 2025年2月28日
feature requesthelp wantedunstale
倉庫指標
- Star
- (80,034 star)
- PR 合併指標
- (平均合併 3天 17小時) (30 天內合併 993 個 PR)
描述
🚀 The feature, motivation and pitch
In V0, we support concurrent partial prefills to avoid TTFT latency with long requests. Implement it in V1
cc @WoosukKwon
Alternatives
No response
Additional context
No response
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.