Fail to move cluster with associated IPAddresses

What steps did you take and what happened?

I was trying to move a cluster from a bootstrap cluster to the new mgmt-cluster, which failed, because of the webhook.

logs:

Creating IPAddress="mgmt-cluster-control-plane-9p4r4" Namespace="default"
Object already exists, updating IPAddress="mgmt-cluster-control-plane-9p4r4" Namespace="default"
Retrying with backoff Cause="error updating \"ipam.cluster.x-k8s.io/v1alpha1, Kind=IPAddress\" default/mgmt-cluster-control-plane-9p4r4: admission webhook \"validation.ipaddress.ipam.cluster.x-k8s.io\" denied the request: spec: Forbidden: the spec of IPAddress is immutable

What did you expect to happen?

I expect the clusterctl move succeed

Cluster API version

v1.5.1

Kubernetes version

v1.26.7

Anything else you would like to add?

Error: [action failed after 10 attempts: error updating "ipam.cluster.x-k8s.io/v1alpha1, Kind=IPAddress" default/mgmt-cluster-control-plane-pwhfv: admission webhook "validation.ipaddress.ipam.cluster.x-k8s.io" denied the request: spec: Forbidden: the spec of IPAddress is immutable, action failed after 10 attempts: error updating "ipam.cluster.x-k8s.io/v1alpha1, Kind=IPAddress" default/mgmt-cluster-control-plane-6xk8h: admission webhook "validation.ipaddress.ipam.cluster.x-k8s.io" denied the request: spec: Forbidden: the spec of IPAddress is immutable, action failed after 10 attempts: error updating "ipam.cluster.x-k8s.io/v1alpha1, Kind=IPAddress" default/mgmt-cluster-control-plane-9p4r4: admission webhook "validation.ipaddress.ipam.cluster.x-k8s.io" denied the request: spec: Forbidden: the spec of IPAddress is immutable]

Label(s) to be applied

/kind bug One or more /area label. See https://github.com/kubernetes-sigs/cluster-api/labels?q=area for the list of labels.