vllm-project/vllm
[Feature]: Remove attention layer name from `unified_kv_cache_update`
开放
#33,267 创建于 2026年1月28日
feature requestgood first issuehelp wantedstalestartup-uxtorch.compile
仓库指标
- 星标
- (80,034 个星标)
- PR 合并指标
- (平均合并 3天 17小时) (30 天内合并 993 个 PR)
描述
🚀 The feature, motivation and pitch
unified_kv_cache_update appears in piecewise cudagraph regions, and each layer has a different name, so each of these have to be compiled separately. This increases cold start compilation time with Dynamo partition because the graphs can no longer be reused.
We should use the same approach as #32805 and #33184 to iterate through the layers in the forward context.
Alternatives
No response
Additional context
No response
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.