kubernetes-sigs/cluster-api

Scale tests show that CAPI or the test itself might leak InMemoryCluster objects

Open

#11,750 opened on Jan 24, 2025

View on GitHub
 (8 comments) (0 reactions) (1 assignee)Go (1,532 forks)auto 404
area/e2e-testinghelp wantedkind/bugpriority/important-soontriage/accepted

Repository metrics

Stars
 (4,267 stars)
PR merge metrics
 (PR metrics pending)

Description

What steps did you take and what happened?

  • Did run the scale-tests using the inmemory provider
  • Noticed there are duplicate inmemorycluster objects created
$ k get inmemorycluster -A | awk '{print $1}'  | sort  | uniq -c | sort -nr | head -n 10
      2 scale-042
      2 scale-035
      2 scale-026
      2 scale-009
      1 scale-500
      1 scale-499
      1 scale-498

What did you expect to happen?

only one inmemorycluster object to exist per cluster.

Cluster API version

~ main

Kubernetes version

No response

Anything else you would like to add?

It is propably due to the scale test which uses CreateOrUpdate.

An update drops all references, so KCP & InMemoryCluster got re-created.

Propably related log when this happens:

failed to create patch helper for Cluster scale-009/scale-009: server side apply dry-run failed for original object: Internal error occurred: failed calling webhook "validation.cluster.cluster.x-k8s.io": failed to call webhook: Post "[https://capi-webhook-service.capi-system.svc:443/validate-cluster-x-k8s-io-v1beta1-cluster?timeout=10s](https://capi-webhook-service.capi-system.svc/validate-cluster-x-k8s-io-v1beta1-cluster?timeout=10s)": context deadline exceeded
error reconciling the Cluster topology
sigs.k8s.io/cluster-api/internal/controllers/topology/cluster.(*Reconciler).reconcile
	/home/ubuntu/go/src/sigs.k8s.io/cluster-api/internal/controllers/topology/cluster/cluster_controller.go:422
sigs.k8s.io/cluster-api/internal/controllers/topology/cluster.(*Reconciler).Reconcile
	/home/ubuntu/go/src/sigs.k8s.io/cluster-api/internal/controllers/topology/cluster/cluster_controller.go:343
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile
	/home/ubuntu/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.4/pkg/internal/controller/controller.go:116
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler
	/home/ubuntu/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.4/pkg/internal/controller/controller.go:303
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem
	/home/ubuntu/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.4/pkg/internal/controller/controller.go:263
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2
	/home/ubuntu/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.4/pkg/internal/controller/controller.go:224
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1700

Label(s) to be applied

/kind bug /area e2e-testing

Contributor guide