[BUG] ModelPool starts a re-activated member while its own previous Pod is still terminating

Open Beginner friendly
#2,578 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
72/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
go, kubernetes

Research direction

Start in internal/controller/modelpool_controller.go at the drain loop around lines 175–224, especially the owner skip at line 186. Check how the loop counts terminating Pods and gates scaleOwner; include the owner’s terminating Pods in that gate. Done when a reactivated member stays at zero replicas until its previous Pods are gone, with a test covering that sequence.

Written by the indexing model from the issue text.

Description

bug

Bug Description

A ModelPool member that just left the slot can be started again while its own previous Pod is still terminating.

The drain loop gates the owner's start on every other member's Pods being gone (othersClear, internal/controller/modelpool_controller.go:175-224 at 2ac4c4476), so the owner never co-resides with an incumbent whose process is still releasing memory. But the loop skips the owner itself (if name == owner { continue }, :186), so the owner's own terminating Pod is never counted. On a unified-memory node that is the co-residence window the comment at :175-182 says the gate exists to close.

Steps to Reproduce

  1. A ModelPool with members A and B, B resident.
  2. B leaves the slot: reclaim drains it for the default, or a swap to A drains it. B's Deployment goes to 0 and its Pod starts terminating while the model is still releasing memory.
  3. A request for B arrives before that Pod is gone. The router activates B (replicas 1).
  4. The next reconcile picks B as owner. The drain loop skips B and finds A with no Pods, so othersClear is true and scaleOwner keeps B at 1.

Expected Behavior

The owner is held at zero replicas until its own terminating Pods are gone too, the same way incumbents are.

Actual Behavior

The Deployment creates a new B Pod next to the old one, which can still hold its memory, so the two co-reside.

Environment

LLMKube Version:

# main at 2ac4c4476

Kubernetes Version:

# n/a (found by reading the code)

Logs

None. Found while reviewing #2540: under its reclaim-down policy, release followed by a request for the same member is the normal pattern, so this would come up more often there.

Dominant language
Go
Stars
232
Forks
37
Avg merge
21h 42m
Merged PRs (30d)
244

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from defilantech/LLMKube

All issues in defilantech/LLMKube

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.