Migration from Manifest to Operator install fails on microk8s; `missing expected volume 'var-run-calico' with hostPath '/var/run/calico'.`
#10,733 opened on Jul 27, 2025
Repository metrics
- Stars
- (7,292 stars)
- PR merge metrics
- (PR metrics pending)
Description
Expected Behavior
I would expect that the migration works.
Current Behavior
The current behavior is a failure:
❯ kubectl describe tigerastatus calico
Name: calico
Namespace:
Labels: <none>
Annotations: <none>
API Version: operator.tigera.io/v1
Kind: TigeraStatus
Metadata:
Creation Timestamp: 2025-07-27T22:23:45Z
Generation: 1
Resource Version: 203984155
UID: a94d79df-1377-4f55-80d0-c2fb7c5fccb2
Spec:
Status:
Conditions:
Last Transition Time: 2025-07-27T22:23:50Z
Message: Existing Calico installation can not be managed by Tigera Operator as it is configured in a way that Operator does not currently support. Please update your existing Calico install config: missing expected volume 'var-run-calico' with hostPath '/var/run/calico'. To fix it, add the expected volume to daemonset/calico-node on daemonset/calico-node
Observed Generation: 1
Reason: MigrationError
Status: True
Type: Degraded
Events: <none>
Context
I am running a small microk8s (v1.33.0) cluster; 3 nodes.
All are amd64 running latest ubuntu 24.04
root@node03:/var/snap/microk8s/current/args/cni-network# microk8s kubectl version
Client Version: v1.33.0
Kustomize Version: v5.6.0
Server Version: v1.33.0
root@node03:/var/snap/microk8s/current/args/cni-network# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 24.04.2 LTS
Release: 24.04
Codename: noble
root@node03:/var/snap/microk8s/current/args/cni-network# uname -a
Linux node03 6.8.0-55-generic #57-Ubuntu SMP PREEMPT_DYNAMIC Wed Feb 12 23:42:21 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
For background, the microk8s project bundles calico and installs it via manifests and then never touches it again. Any updates / configuration tweaks are done via changes to the manifests and then a re-apply.
I have been doing updates from time to time by patching the image: ... to the latest version of calico and that's been fine.
Just a few hours ago I migrated to 3.30.2 from 3.28.1.
I am currently trying to migrate to the Operator based pattern so I have additional CRDs at my disposal.
Unfortunately, the migration has failed with the above error.
How I got here:
❯ kubectl apply --server-side --force-conflicts -f https://raw.githubusercontent.com/projectcalico/calico/v3.30.2/manifests/operator-crds.yaml
customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io serverside-applied
<...>
customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org serverside-applied
customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org serverside-applied
<...>
customresourcedefinition.apiextensions.k8s.io/baselineadminnetworkpolicies.policy.networking.k8s.io serverside-applied
❯ kubectl apply --server-side --force-conflicts -f https://raw.githubusercontent.com/projectcalico/calico/v3.30.2/manifests/tigera-operator.yaml
namespace/tigera-operator serverside-applied
serviceaccount/tigera-operator serverside-applied
clusterrole.rbac.authorization.k8s.io/tigera-operator-secrets serverside-applied
clusterrole.rbac.authorization.k8s.io/tigera-operator serverside-applied
clusterrolebinding.rbac.authorization.k8s.io/tigera-operator serverside-applied
rolebinding.rbac.authorization.k8s.io/tigera-operator-secrets serverside-applied
deployment.apps/tigera-operator serverside-applied
❯ kubectl create -f - <<EOF
apiVersion: operator.tigera.io/v1
kind: Installation
metadata:
name: default
spec: {}
EOF
installation.operator.tigera.io/default created
And then the very next command in shell history is describe tigerastatus calico which gives me the error about /var/run/calico not being as expected.
But here's the kicker, the existing CNI config does have a var-run-calico volume defined:
root@node03:/var/snap/microk8s/current/args/cni-network# cat cni.yaml | grep -b5 var-run-calico
359397- # name: sys-fs
359437- # # Bidirectional is required to ensure that the new mount we make at /sys/fs/bpf propagates to the host
359567- # # so that it outlives the init container.
359636- # mountPropagation: Bidirectional
359695- # - mountPath: /var/run/calico
359749: # name: var-run-calico
359797- # # Bidirectional is required to ensure that the new mount we make at /run/calico/cgroup propagates to the host
359934- # # so that it outlives the init container.
360003- # mountPropagation: Bidirectional
360062- # # Mount /proc/ from host which usually is an init program at /nodeproc. It's needed by mountns binary,
360190- # # executed by calico-node, to mount root cgroup2 fs at /run/calico/cgroup to attach CTLB programs correctly.
--
366666- readOnly: true
366705- - mountPath: /run/xtables.lock
366758- name: xtables-lock
366801- readOnly: false
366841- - mountPath: /var/run/calico
366892: name: var-run-calico
366937- readOnly: false
366977- - mountPath: /var/lib/calico
367028- name: var-lib-calico
367073- readOnly: false
367113- - name: policysync
--
367593- volumes:
367614- # Used by calico-node.
367653- - name: lib-modules
367689- hostPath:
367717- path: /lib/modules
367758: - name: var-run-calico
367797- hostPath:
367825- path: /var/snap/microk8s/current/var/run/calico
367895- - name: var-lib-calico
367934- hostPath:
367962- path: /var/snap/microk8s/current/var/lib/calico
And both /var/run/calico and /var/snap/microk8s/current/var/run/calico exist:
root@node03:/var/snap/microk8s/current/args/cni-network# ls -lah /var/snap/microk8s/current/var/run/calico
total 16K
drwxr-xr-x 4 root root 4.0K Jul 27 13:36 .
drwxr-xr-x 4 root root 4.0K Mar 4 2024 ..
drwx------ 2 root root 4.0K Mar 4 2024 cgroup
drwxr-xr-x 2 root root 4.0K Jul 27 15:10 endpoint-status
root@node03:/var/snap/microk8s/current/args/cni-network# ls -lah /var/run/calico/
total 0
drwxr-xr-x 2 root root 60 Jul 27 12:59 .
drwxr-xr-x 33 root root 1.1K Jul 27 12:59 ..
-rw------- 1 root root 0 Jul 27 12:59 ipam.lock
It's not clear to me what the issue is or if there's a setting in the kind: Installation that would let me adjust where the operator thinks var-run-calico should be?