[BUG] Docs and comments describe InsufficientMemory only as a start refusal, but a serving engine can now carry it
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 25/100
- Issue type
- Documentation
- Clarity
- Clearly specified
- Activity status
- Stale
- Tech stack
- go
- Domain
- documentation
Research direction
Read the listed comments in api/v1alpha1/constants.go and internal/controller/scheduling.go, then the memory guidance in docs/site/guides/macos-metal.md and docs/site/guides/memory-pressure.md. Update the descriptions to cover both refused starts and serving engines over budget, including how to distinguish and handle each; preserve extension-less links in docs/site. Done when comments and docs match the behavior described in the issue.
Written by the indexing model from the issue text.
Description
Bug Description
After #2595, the metal agent also writes status.schedulingStatus: InsufficientMemory on an engine that is serving. When an adopted engine's re-estimated charge doesn't fit, the status reads adopted engine is serving over its memory budget: ... and one Warning Event goes out. The engine keeps running, and the agent clears the status once the charge fits again, including after the Model's memoryBudget is raised.
The code comments and docs still describe InsufficientMemory only as a refusal to start:
api/v1alpha1/constants.go:245-258: the refusal reasons are "the values the metal-agent writes when it declines to start a service".internal/controller/scheduling.go:60-62:agentRefusalReasonsare the agent's "refusals to start a service".docs/site/guides/macos-metal.md:786-794: "the agent refuses to start it and marks the InferenceService withstatus.schedulingStatus: InsufficientMemory".docs/site/guides/memory-pressure.md:30-37: the section "Before a start: the memory budget" describes the same refusal only.
Line numbers are from #2595's head (fad1c80a).
Steps to Reproduce
- On a Mac, serve a model whose Model sets
hardware.memoryBudgetbelow what the current estimate says the engine needs. The #2535 case gets there by an agent upgrade that raises the estimate. - Restart the metal agent, so it adopts the running engine.
- The InferenceService reads
InsufficientMemorywith the messageadopted engine is serving over its memory budget: ...while the engine serves. The docs say this status means the model was refused and is not running.
Expected Behavior
The comments and docs describe both meanings of InsufficientMemory:
- A refused start: the service is not running, and the agent re-checks it on a backoff until it fits.
- A serving engine over its budget: the engine keeps serving, one Warning Event goes out, and the agent clears the status once the charge fits again (memory freed elsewhere, or the Model's budget raised).
The docs also say how to tell the two apart (the adopted engine is serving over its memory budget: prefix) and what to do about each. Nothing is stopped in the second case.
Actual Behavior
Only the start refusal is described.
Environment
LLMKube Version:
llmkube version
# Output: main after #2595 merges (head fad1c80a)
Kubernetes Version:
kubectl version
# Output: N/A (docs and comments)
Cluster Type:
- Other: any cluster with a Metal agent
GPU (if applicable):
- Metal (Apple Silicon)
Logs
Controller Logs:
N/A
Inference Pod Logs:
N/A
Additional Context
- Docs and code comments only; no behavior change. Wait for #2595 to merge, since the text describes its behavior.
- Links inside
docs/sitestay extension-less. - Related: #2535, #2595.
- Dominant language
- Go
- Stars
- 232
- Forks
- 37
- Avg merge
- 21h 42m
- Merged PRs (30d)
- 244
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- Ships a Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from defilantech/LLMKube
-
component/metal-agent documentation
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
defilantech/LLMKube#2581 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
defilantech/LLMKube#2578 ·
Maintainers usually reply within 1 day
-
documentation helm priority/low size/small
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
defilantech/LLMKube#2420 ·
Maintainers usually reply within 1 day
-
component/metal-agent enhancement priority/low size/medium
Difficulty 4/5 3-5 days Newbie friendliness 48/100
defilantech/LLMKube#2597 ·
Maintainers usually reply within 1 day
-
kind/cleanup size/large
Difficulty 5/5 Over a week Newbie friendliness 15/100
defilantech/LLMKube#2593 ·
Maintainers usually reply within 1 day
All issues in defilantech/LLMKube
Similar issues
-
☢️ Bug v3
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
area/import-export area/projects concern/ux
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
go-vikunja/vikunja#4455 · 1 comment ·
Maintainers usually reply within 2 days
-
Dependabot PRs no longer merge automaticallyPossibly taken @dwisiswant0 claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
projectdiscovery/urlfinder#160 ·
Maintainers usually reply within 1 day
-
database
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
chnm/legal-modernism#450 ·
Maintainers usually reply within 1 day
-
tech-debt
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 1 day