[FEATURE] Stage nfs:// sources into the model cache
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 52/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- go, kubernetes
- Domain
- ai, infrastructure
Research direction
Start by reading the existing source handling in buildPVCStorageConfig, buildModelInitCommand, and buildMultiFileInitCommand, plus the corresponding prefetch Job path. Trace how s3:// sources provide file lists, cache locking, verification, and publishing, then compare the local source behavior. Done means an NFS source can stage listed files and mmproj into the cache for both model startup and prefetch, with the described refresh and checksum behavior.
Written by the indexing model from the issue text.
Description
Feature Description
Add an nfs:// model source that the model downloader copies into the model cache, the same way it stages hf://, https://, and s3:// sources today. The NFS export stays the source of truth, and the cache PVC holds a local copy after the first pull.
Problem Statement
As a homelab operator with all my models on a NAS, I want the NAS to be the cold tier and a local cache PVC to be the hot tier, so that only the first load of a model reads from the NAS.
Today a model on NFS cannot reach the cache:
pvc://on an NFS-backed PersistentVolume.buildPVCStorageConfigmounts the claim read-only into the serving pod and never stages it, so every pod start reads the weights from the NAS.- A local or
file://path is copied into the cache (thecp /host-model/model.ggufbranch ofbuildModelInitCommand), but only from a hostPath on the node, gated by the allowlist from GHSA-jw3m-8q7m-f35r. That needs the export mounted on the host, which an immutable OS such as Talos does not do. The copy also covers a single file only:buildMultiFileInitCommandhas no local branch, so a model with anmmprojcannot use it.
The result: the NAS is a runtime dependency for every pod start. A cold start reads the full GGUF from the NAS, which is HDD mirrors in my case. When the NAS is down, no model can start, even though the bytes could be on local NVMe.
The workaround is to run an S3 gateway on the NAS so the downloader treats it as remote. That adds a service on the NAS only to get caching, and it is one more component that can fail.
Proposed Solution
A new nfs://<server>/<export path> scheme that reuses the existing local-copy path, with an nfs volume source in place of the hostPath:
- Only the prefetch Job gets a read-only
nfsvolume for the export, and copies the listed files into the cache directory. Cache key,flockon.llmkube-download.lock, the completion marker, and the publish step work as they do for the local copy today. filesandmmprojwork as they do for ans3://prefix. This needs a copy branch inbuildMultiFileInitCommand, which would also give multi-file staging to local sources. Explicit file names are enough for a first version: the controller does not mount the export, so it cannot expand globs or read the GGUF header itself.refreshPolicy: OnChangecompares size and mtime, which the docs already describe for local sources. The s3 single-file path has no OnChange today, so this is new code too.sha256/fileSha256verify the copy before it becomes the cache.prefetch: truestages it ahead of any InferenceService.s3://is not prefetchable today, so this needs its own prefetch branch rather than reusing the s3 one.
apiVersion: inference.llmkube.dev/v1alpha1
kind: Model
metadata:
name: qwen3-8-27b
spec:
source: nfs://tank.internal/mnt/home-pool/models/Swift-1.5-Qwen3.8-27B
files:
- Swift-1.5-Qwen3.8-27B-NVFP4.gguf
mmproj: Swift-1.5-Qwen3.8-27B-mmproj-F16.gguf
refreshPolicy: OnChange
prefetch: true
The NAS is cold storage only. The serving pod must never reference the NFS export: the kubelet mounts every volume in the pod spec before any container starts, so an init container that finds the cache already full would still need the NAS reachable to start the pod. Staging should therefore happen only in the prefetch Job (a separate pod), and the serving pod should mount just the cache PVC and wait until the cache is ready. Today the downloader runs as an init container in the serving pod, so this needs nfs:// to always go through prefetch, which s3:// does not support yet either.
Two constraints on an inline nfs volume:
- Pod Security Admission
restricteddoes not allownfsvolumes, so a namespace onrestrictedcannot run the prefetch Job with this scheme. - An inline
nfsvolume takes nomountOptions(nfsvers,nconnect,hard/soft), which a PV does.
The downloader runs as uid 101, so the export has to be readable by that uid (or world-readable). If you want the same posture as local paths, an operator flag that allowlists NFS servers or export roots would fit, empty by default.
Alternatives Considered
pvc://with an opt-instage: true. This generalizes the feature to any claim (NFS, CephFS, a local disk), at the cost of a PV and PVC per export. Because the claim is a PVC, it passes PSArestricted, keeps the PV'smountOptions, and needs no new allowlist. Given the constraints above this may be the better shape, and I would be happy with either.- An S3 gateway on the NAS. It works today, but it adds a service only to trigger caching.
pvc://on NFS plus the node page cache. No control over eviction, and the NAS is still required at every pod start.
Open questions
nfs://orpvc://+stage: true?- Staging only through the prefetch Job, with the serving pod gated on cache readiness: does that fit how you want prefetch to evolve?
- Should
stagealso cover multi-file local sources (buildMultiFileInitCommandhas no local branch)?
Additional Context
- Related issues: #52 (persistent model storage), #1353 (cache PVC provisioned for
pvc://sources), #1762 (resumable downloads; a copy from NFS would avoid that problem for large files) - Similar features in other projects: none that I know of for NFS-to-local tiering of model weights.
- Workarounds you're currently using: copying models between the NAS and a local PVC by hand.
Priority
- Critical - Blocking my use case
- High - Would significantly improve my workflow
- Medium - Nice to have
- Low - Minor improvement
Willingness to Contribute
- Yes, I can submit a PR
- Yes, I can help test
- No, but I can provide feedback
- No, just requesting
Assisted by Claude Code. The investigation and findings were reviewed and verified by the author.
- Dominant language
- Go
- Stars
- 232
- Forks
- 37
- Avg merge
- 21h 42m
- Merged PRs (30d)
- 244
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- Ships a Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from defilantech/LLMKube
-
component/metal-agent documentation
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
defilantech/LLMKube#2581 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
defilantech/LLMKube#2578 ·
Maintainers usually reply within 1 day
-
documentation helm priority/low size/small
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
defilantech/LLMKube#2420 ·
Maintainers usually reply within 1 day
-
component/metal-agent enhancement priority/low size/medium
Difficulty 4/5 3-5 days Newbie friendliness 48/100
defilantech/LLMKube#2597 ·
Maintainers usually reply within 1 day
-
[BUG] Docs and comments describe InsufficientMemory only as a start refusal, but a serving engine can now carry itPossibly taken @defilan-agent claimed this today. Openbug component/metal-agent documentation priority/low size/small
Difficulty 2/5 1-3 hours Newbie friendliness 25/100
defilantech/LLMKube#2596 ·
Maintainers usually reply within 1 day
All issues in defilantech/LLMKube
Similar issues
-
☢️ Bug v3
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
area/import-export area/projects concern/ux
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
go-vikunja/vikunja#4455 · 1 comment ·
Maintainers usually reply within 2 days
-
Dependabot PRs no longer merge automaticallyPossibly taken @dwisiswant0 claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
projectdiscovery/urlfinder#160 ·
Maintainers usually reply within 1 day
-
database
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
chnm/legal-modernism#450 ·
Maintainers usually reply within 1 day
-
tech-debt
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 1 day