[BUG] ModelPool starts a re-activated member while its own previous Pod is still terminating
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 72/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- go, kubernetes
- Domain
- backend, distributed-systems
Research direction
Start in internal/controller/modelpool_controller.go at the drain loop around lines 175–224, especially the owner skip at line 186. Check how the loop counts terminating Pods and gates scaleOwner; include the owner’s terminating Pods in that gate. Done when a reactivated member stays at zero replicas until its previous Pods are gone, with a test covering that sequence.
Written by the indexing model from the issue text.
Description
Bug Description
A ModelPool member that just left the slot can be started again while its own previous Pod is still terminating.
The drain loop gates the owner's start on every other member's Pods being gone (othersClear, internal/controller/modelpool_controller.go:175-224 at 2ac4c4476), so the owner never co-resides with an incumbent whose process is still releasing memory. But the loop skips the owner itself (if name == owner { continue }, :186), so the owner's own terminating Pod is never counted. On a unified-memory node that is the co-residence window the comment at :175-182 says the gate exists to close.
Steps to Reproduce
- A ModelPool with members A and B, B resident.
- B leaves the slot:
reclaimdrains it for the default, or a swap to A drains it. B's Deployment goes to 0 and its Pod starts terminating while the model is still releasing memory. - A request for B arrives before that Pod is gone. The router activates B (replicas 1).
- The next reconcile picks B as owner. The drain loop skips B and finds A with no Pods, so
othersClearis true andscaleOwnerkeeps B at 1.
Expected Behavior
The owner is held at zero replicas until its own terminating Pods are gone too, the same way incumbents are.
Actual Behavior
The Deployment creates a new B Pod next to the old one, which can still hold its memory, so the two co-reside.
Environment
LLMKube Version:
# main at 2ac4c4476
Kubernetes Version:
# n/a (found by reading the code)
Logs
None. Found while reviewing #2540: under its reclaim-down policy, release followed by a request for the same member is the normal pattern, so this would come up more often there.
- Dominant language
- Go
- Stars
- 232
- Forks
- 37
- Avg merge
- 21h 42m
- Merged PRs (30d)
- 244
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- Ships a Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from defilantech/LLMKube
-
component/metal-agent documentation
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
defilantech/LLMKube#2581 ·
Maintainers usually reply within 1 day
-
documentation helm priority/low size/small
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
defilantech/LLMKube#2420 ·
Maintainers usually reply within 1 day
-
component/metal-agent enhancement priority/low size/medium
Difficulty 4/5 3-5 days Newbie friendliness 48/100
defilantech/LLMKube#2597 ·
Maintainers usually reply within 1 day
-
[BUG] Docs and comments describe InsufficientMemory only as a start refusal, but a serving engine can now carry itPossibly taken @defilan-agent claimed this today. Openbug component/metal-agent documentation priority/low size/small
Difficulty 2/5 1-3 hours Newbie friendliness 25/100
defilantech/LLMKube#2596 ·
Maintainers usually reply within 1 day
-
kind/cleanup size/large
Difficulty 5/5 Over a week Newbie friendliness 15/100
defilantech/LLMKube#2593 ·
Maintainers usually reply within 1 day
All issues in defilantech/LLMKube
Similar issues
-
☢️ Bug v3
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
area/import-export area/projects concern/ux
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
go-vikunja/vikunja#4455 · 1 comment ·
Maintainers usually reply within 2 days
-
Dependabot PRs no longer merge automaticallyPossibly taken @dwisiswant0 claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
projectdiscovery/urlfinder#160 ·
Maintainers usually reply within 1 day
-
database
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
chnm/legal-modernism#450 ·
Maintainers usually reply within 1 day
-
tech-debt
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 1 day