docs: add LocalModelCache support for LLMInferenceService
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 78/100
- Issue type
- Documentation
- Clarity
- Clearly specified
- Activity status
- Quiet
- Tech stack
- kubernetes
- Domain
- documentation
Research direction
Update docs/model-serving/generative-inference/modelcache/localmodel.md, using the related PR #5318 as context for LocalModelCache support with LLMInferenceService. Document webhook defaults, URI rewriting, delete protection, label propagation, and YAML examples for LocalModelService and LocalModelNamespaceCache. Done means the existing documentation covers each listed behavior and example.
Written by the indexing model from the issue text.
Description
Summary
KServe v0.19 added LocalModelCache support for LLMInferenceService, but the existing LocalModelCache documentation (docs/model-serving/generative-inference/modelcache/localmodel.md) only covers usage with InferenceService.
What needs to be documented
- How LocalModelCache integrates with LLMInferenceService
- Webhook defaulter behavior: automatic label/annotation setting when model URI matches a cache
- URI rewriting from
hf://modeltopvc://<cached-pvc>/models/<storageKey>/ - Delete protection: cache deletion is blocked when active LLMInferenceServices reference it
- Label propagation to Deployment and LeaderWorkerSet pod templates
- Example YAML for LLMInferenceService using LocalModelCache and LocalModelNamespaceCache
Related PR
Since
KServe v0.19
- Dominant language
- MDX
- Stars
- 113
- Forks
- 193
- Avg merge
- 5d 20h
- Merged PRs (30d)
- 3
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from kserve/website
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-2 days Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Similar issues
-
user-reported
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Kong/developer.konghq.com#7316 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
HarperFast/skills#96 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
infinispan/infinispan#18150 ·
-
bug triage:deciding
Difficulty 1/5 Under an hour Newbie friendliness 88/100
open-telemetry/otel-arrow#4132 ·
-
Ecosystem: ClawMetry — the Qwen Code reader is now free and open source (follow-up to #9294 / #9338) Opencategory/integration priority/P3 scope/documentation status/ready-for-human type/feature-request
Difficulty 1/5 Under an hour Newbie friendliness 84/100