LMCache/LMCache

KV-Cache Persistence Issue with SSD Offloading

オープン

#1,175 opened on 2025/07/28

 (12 件のコメント) (0 件のリアクション) (0 人の担当者)Python (1,195 件のフォーク)user submission
good first issuehelp wanted

Repository metrics

Stars
 (8,345 個のスター)
PR merge metrics
 (平均マージ 6d 13h) (30d で 204 merged PRs)

説明

Description:
follow the SSD offload guide in :

https://docs.lmcache.ai/kv_cache/local_storage.html

experimentation with vLLM and LMCache, we observed that the KV-Cache offloading to SSD works partially but lacks persistence across server restarts.

Steps to Reproduce:

  1. Configure vLLM server with SSD offloading for KV-Cache
  2. Run inference scripts with a specific prompt
  3. Observe 2.4GB .pt file generated in the SSD cache directory
  4. Restart the vLLM server process
  5. Query the same prompt again
  6. the kv-cache regenerated in SSD

Expected Behavior:
The existing KV-Cache files should be reused, avoiding redundant computation and disk I/O.

Actual Behavior:
New identical KV-Cache files are regenerated instead of reusing the existing ones

This optimization would significantly improve latency for recurring queries in production scenarios. I would appreciate guidance on recommended methods to achieve proper KV-Cache persistence across server restarts in this workflow.

コントリビューターガイド