Kubernetes reference chart: an operator command can leave an unowned NetworkPolicy when the policy create times out

Open Beginner friendly
#2,761 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
62/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
kubernetes, python

Research direction

Start in deploy/stack_operator.py at Installation.job() and follow the order of the NetworkPolicy create, the Job create and the assignment of active_job; then find the path that releases the Lease on failure. Look for existing operator tests around job creation to see how kubectl calls are stubbed, since you will want a failing-policy case. The fix is to either attempt a kubectl delete networkpolicy --ignore-not-found before releasing the Lease or keep the Lease and report which object to check, so done means the uncertain-create path leaves no unowned policy and the failure is reported.

Written by the indexing model from the issue text.

Description

Installation.job() in deploy/stack_operator.py creates an operation Job's NetworkPolicy before it sets active_job. If kubectl create for the policy fails with an uncertain outcome, such as a server timeout or a lost connection, the operator releases its Lease. The policy may still exist.

That policy has no owner reference yet, because the owner is set to the Job only after the Job is created, and it has no TTL. So it stays until someone deletes it by hand.

Impact: low. The policy selects only the unique registrystack.org/operation-id label of an operation that never ran, so it affects no other workload. It is clutter, and it is left over after an operator command that did not report it.

Possible fixes:

  • On a failed policy create, try once to kubectl delete networkpolicy <name> --ignore-not-found before releasing the Lease, and report if that delete also fails.
  • Or treat an uncertain policy create like an uncertain Job create: keep the Lease and tell the operator which object to check.

Found during the review of #2758. It is older than #2758 and was left out of that PR.

Dominant language
Rust
Stars
2
Forks
0
Avg merge
9h 48m
Merged PRs (30d)
151

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from registrystack/registry-stack

All issues in registrystack/registry-stack

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.