kubernetes-sigs/cluster-api
View on GitHubScale tests show that CAPI or the test itself might leak InMemoryCluster objects
Open
#11,750 opened on Jan 24, 2025
area/e2e-testinghelp wantedkind/bugpriority/important-soontriage/accepted
Repository metrics
- Stars
- (4,267 stars)
- PR merge metrics
- (PR metrics pending)
Description
What steps did you take and what happened?
- Did run the scale-tests using the inmemory provider
- Noticed there are duplicate inmemorycluster objects created
$ k get inmemorycluster -A | awk '{print $1}' | sort | uniq -c | sort -nr | head -n 10
2 scale-042
2 scale-035
2 scale-026
2 scale-009
1 scale-500
1 scale-499
1 scale-498
What did you expect to happen?
only one inmemorycluster object to exist per cluster.
Cluster API version
~ main
Kubernetes version
No response
Anything else you would like to add?
It is propably due to the scale test which uses CreateOrUpdate.
An update drops all references, so KCP & InMemoryCluster got re-created.
Propably related log when this happens:
failed to create patch helper for Cluster scale-009/scale-009: server side apply dry-run failed for original object: Internal error occurred: failed calling webhook "validation.cluster.cluster.x-k8s.io": failed to call webhook: Post "[https://capi-webhook-service.capi-system.svc:443/validate-cluster-x-k8s-io-v1beta1-cluster?timeout=10s](https://capi-webhook-service.capi-system.svc/validate-cluster-x-k8s-io-v1beta1-cluster?timeout=10s)": context deadline exceeded
error reconciling the Cluster topology
sigs.k8s.io/cluster-api/internal/controllers/topology/cluster.(*Reconciler).reconcile
/home/ubuntu/go/src/sigs.k8s.io/cluster-api/internal/controllers/topology/cluster/cluster_controller.go:422
sigs.k8s.io/cluster-api/internal/controllers/topology/cluster.(*Reconciler).Reconcile
/home/ubuntu/go/src/sigs.k8s.io/cluster-api/internal/controllers/topology/cluster/cluster_controller.go:343
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile
/home/ubuntu/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.4/pkg/internal/controller/controller.go:116
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler
/home/ubuntu/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.4/pkg/internal/controller/controller.go:303
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem
/home/ubuntu/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.4/pkg/internal/controller/controller.go:263
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2
/home/ubuntu/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.4/pkg/internal/controller/controller.go:224
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1700
Label(s) to be applied
/kind bug /area e2e-testing