Bug: v1.6's delete-protection finalizer is never removed from migrated TemporalWorkerDeployment/TemporalConnection, so they stay Terminating after the v1.7 migration
Los mantenedores suelen responder en 1 día
@nmechack27 ya está trabajando en esto.
Desde el 12/10/2026.
Evaluación
- Dificultad
- 3/5
- Tiempo estimado
- 1-2 días
- Aptitud para principiantes
- 57/100
Línea de trabajo
Start by reading deprecated_reconciler.go and trace how the deprecated reconcilers release temporal.io/migration-guard; compare that with the v1.6 finalizer behavior described in the issue. Check the migration tests and add coverage for migrated objects that still have temporal.io/delete-protection. Done means the old objects can finish deletion after ownership transfers, without running server-side cleanup on them.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Summary
After migrating from v1.6 to v1.7 (TemporalWorkerDeployment → WorkerDeployment, TemporalConnection → Connection), the old objects stay in Terminating forever. The migration itself succeeds: the new WorkerDeployment adopts the Deployments, the old object is labelled migrated, and the deprecated reconciler releases temporal.io/migration-guard. But the old objects still carry the temporal.io/delete-protection finalizer that v1.6 added, and nothing in v1.7+ ever removes it from the deprecated kinds.
Versions
- Controller: v1.6.0 (chart 0.25.0) → v1.7.1 (chart 0.28.1), CRDs chart upgraded first as documented. The same code is still present in v1.12.0.
- Applied with GitOps (Argo CD); same steps as docs/migration-crd-rename.md Step 3: the manifests change kind, so the old objects are deleted and the new ones created in one sync.
Steps to reproduce
- Run v1.6.0 with a
TemporalWorkerDeploymentand aTemporalConnection. v1.6'sWorkerDeploymentReconcileraddstemporal.io/delete-protectionto both (worker_controller.go,finalizerName). - Upgrade the CRDs, then the controller to v1.7.x.
- Replace the manifests with
WorkerDeployment/Connectionof the same names (this deletes the old objects).
Expected
Per the migration guide: "The migration-guard finalizer holds each deprecated resource in Terminating state until the controller confirms ownership transfer is complete, then removes the finalizer and lets the deletion finish."
Actual
The WorkerDeployments and Connection reconcile and the workers keep running on the new objects, but the old objects never finish deleting:
$ kubectl get temporalworkerdeployment <name> -o jsonpath='{.metadata.finalizers}'
["temporal.io/delete-protection"]
migration-guard is gone (the migrated label is set), and only delete-protection remains. For an Argo CD application this blocks the sync: the old objects never go away, so the app stays out of sync and a release that waits on it times out.
Cause
From v1.7 the deprecated kinds are handled by DeprecatedTWDReconciler / DeprecatedTCReconciler, which only add and remove temporal.io/migration-guard. The comment in deprecated_reconciler.go says that is deliberate ("intentionally distinct from the existing temporal.io/delete-protection finalizer, which lives on WorkerDeployment/Connection"). That reasoning holds for objects created after v1.7. But every object that existed under v1.6 already carries delete-protection, and since the WorkerDeploymentReconciler no longer reconciles the old kinds, nothing removes it.
Workaround
Once migration is confirmed, remove the leftover finalizer, guarded so that it only acts when delete-protection is the last finalizer left (that is, after migration-guard was released):
kubectl -n <ns> patch temporalworkerdeployment <name> --type=json -p='[
{"op":"test","path":"/metadata/finalizers","value":["temporal.io/delete-protection"]},
{"op":"remove","path":"/metadata/finalizers"}]'
Same for temporalconnection. This made no Temporal API call: the server-side Worker Deployment stays owned by the new WorkerDeployment, and the Kubernetes Deployments (owned only by the new object by then) are untouched.
Suggested fix
When the deprecated reconcilers release migration-guard on a migrated object, also remove temporal.io/delete-protection from it. The new WorkerDeployment / Connection already owns the server-side cleanup that finalizer was for, so this cleanup must not run for the old object. Alternatively, document the step in the migration guide, in the same way the downgrade guide already covers re-adding delete-protection.
- Lenguaje dominante
- Go
- Estrellas
- 197
- Forks
- 82
- Merge medio
- 3 d 40 min
- PR fusionados (30 d)
- 35
Preparar el entorno
- Incluye un Dockerfile o un archivo de Docker Compose
- Sin plantilla de pull request
- Sin guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de temporalio/temporal-worker-controller
-
[Feature Request] Configure temporal-worker-controller to use hostNetworkPosiblemente ocupada @josh-ferrell la tomó hace 3 días. Abiertoenhancement
Dificultad 4/5 3-5 días Aptitud para principiantes 35/100
temporalio/temporal-worker-controller#680 ·
Los mantenedores suelen responder en 1 día
-
[Feature Request] Helm chart: support topologySpreadConstraints on the controller DeploymentPosiblemente ocupada @akolacz la tomó hace 11 días. Abiertohelm
temporalio/temporal-worker-controller#658 · 1 reacción · 1 asignado ·
Los mantenedores suelen responder en 1 día
-
rbac
Dificultad 4/5 3-5 días Aptitud para principiantes 48/100
temporalio/temporal-worker-controller#654 ·
Los mantenedores suelen responder en 1 día
-
AI agent skillsAbierto
Dificultad 3/5 1-2 días Aptitud para principiantes 68/100
temporalio/temporal-worker-controller#633 · 1 reacción ·
Los mantenedores suelen responder en 1 día
-
refactor integration testsAbierto
Dificultad 4/5 3-5 días Aptitud para principiantes 62/100
temporalio/temporal-worker-controller#632 · 2 comentarios ·
Los mantenedores suelen responder en 1 día
Todos los issues de temporalio/temporal-worker-controller
Issues similares
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
cloudevents/sdk-go#1346 ·
-
needs-triage
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
kubernetes/cloud-provider-aws#1517 · 1 comentario ·
Los mantenedores suelen responder en 1 día
-
agent-butler-finding chore
Dificultad 1/5 Menos de una hora Aptitud para principiantes 88/100
jordansmall/spindrift#6221 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
Los mantenedores suelen responder en 3 días