[BUG] Docs and comments describe InsufficientMemory only as a start refusal, but a serving engine can now carry it

Open
#2,596 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@defilan-agent is already working on this.

Since Nov 20, 2026.

  • #2600 by @defilan-agent — open

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
25/100
Issue type
Documentation
Clarity
Clearly specified
Activity status
Stale
Tech stack
go
Domain
documentation

Research direction

Read the listed comments in api/v1alpha1/constants.go and internal/controller/scheduling.go, then the memory guidance in docs/site/guides/macos-metal.md and docs/site/guides/memory-pressure.md. Update the descriptions to cover both refused starts and serving engines over budget, including how to distinguish and handle each; preserve extension-less links in docs/site. Done when comments and docs match the behavior described in the issue.

Written by the indexing model from the issue text.

Description

bug component/metal-agent documentation priority/low size/small

Bug Description

After #2595, the metal agent also writes status.schedulingStatus: InsufficientMemory on an engine that is serving. When an adopted engine's re-estimated charge doesn't fit, the status reads adopted engine is serving over its memory budget: ... and one Warning Event goes out. The engine keeps running, and the agent clears the status once the charge fits again, including after the Model's memoryBudget is raised.

The code comments and docs still describe InsufficientMemory only as a refusal to start:

  • api/v1alpha1/constants.go:245-258: the refusal reasons are "the values the metal-agent writes when it declines to start a service".
  • internal/controller/scheduling.go:60-62: agentRefusalReasons are the agent's "refusals to start a service".
  • docs/site/guides/macos-metal.md:786-794: "the agent refuses to start it and marks the InferenceService with status.schedulingStatus: InsufficientMemory".
  • docs/site/guides/memory-pressure.md:30-37: the section "Before a start: the memory budget" describes the same refusal only.

Line numbers are from #2595's head (fad1c80a).

Steps to Reproduce

  1. On a Mac, serve a model whose Model sets hardware.memoryBudget below what the current estimate says the engine needs. The #2535 case gets there by an agent upgrade that raises the estimate.
  2. Restart the metal agent, so it adopts the running engine.
  3. The InferenceService reads InsufficientMemory with the message adopted engine is serving over its memory budget: ... while the engine serves. The docs say this status means the model was refused and is not running.

Expected Behavior

The comments and docs describe both meanings of InsufficientMemory:

  • A refused start: the service is not running, and the agent re-checks it on a backoff until it fits.
  • A serving engine over its budget: the engine keeps serving, one Warning Event goes out, and the agent clears the status once the charge fits again (memory freed elsewhere, or the Model's budget raised).

The docs also say how to tell the two apart (the adopted engine is serving over its memory budget: prefix) and what to do about each. Nothing is stopped in the second case.

Actual Behavior

Only the start refusal is described.

Environment

LLMKube Version:

llmkube version
# Output: main after #2595 merges (head fad1c80a)

Kubernetes Version:

kubectl version
# Output: N/A (docs and comments)

Cluster Type:

  • Other: any cluster with a Metal agent

GPU (if applicable):

  • Metal (Apple Silicon)

Logs

Controller Logs:

N/A

Inference Pod Logs:

N/A

Additional Context

  • Docs and code comments only; no behavior change. Wait for #2595 to merge, since the text describes its behavior.
  • Links inside docs/site stay extension-less.
  • Related: #2535, #2595.
Dominant language
Go
Stars
232
Forks
37
Avg merge
21h 42m
Merged PRs (30d)
244

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from defilantech/LLMKube

All issues in defilantech/LLMKube

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.