bug: pod spec doesn't match sts template spec
I maintainer di solito rispondono entro 1 giorno
@razvan ci sta già lavorando.
Dal 3/3/2026.
Valutazione
Questa issue non è ancora stata valutata.
Descrizione
Workaround
Ensure catalog definitions are created before creating the Trino cluster.
Proper solution is probably to have a separate controller for the catalogs.
Tasks
- ...
- Revert https://github.com/stackabletech/trino-operator/pull/857, so we test the "normal" way of creating TrinoClusters before TrinoCatalogs again.
- Revert https://github.com/stackabletech/nifi-operator/pull/907, so we test the "normal" way of creating TrinoClusters before TrinoCatalogs again.
Description
TL;DR: The tests create the Trino cluster and then the iceberg catalog definition. In the first reconciliation pass, the operator doesn't find this catalog so it applies a StatefulSet version without the necessary export statements. In a second pass, the operator updates the STS definition but the Pod is not updated. This new behaviour is caused by the MaxUnavailableStatefulSet feature gate in Kubernetes 1.35.
Upsteam issue to bring more clarity: https://github.com/kubernetes/kubernetes/issues/137409
Update 3: The beta feature alone is responsible for this bug. It is present even if the sts/pods are not annotated by other controllers (I uninstalled commons and secret from the cluster).
Update 2: The beta feature MaxUnavailableStatefulSet (enabled by default in v1.35.0) is responsible for this bug. See More Tests below.
Update: Apparently this bug depends on the Kubernetes version. See More Tests below.
In some circumstances the pod spec of Trino pods (coordinator and worker) do not match the template spec of the corresponding StatefulSet objects.
Currently one pod field is known to be different, and this is the container args field.
This was discovered while investigating the failure of the opa-authorization integration tests.
In this test, the Trino pods fail to start. The error message is:
trino java.lang.RuntimeException: Environment variable is not set: CATALOG_ICEBERG_S3_AWS_ACCESS_KEY
trino at io.airlift.configuration.secrets.SecretsResolver.lambda$getResolvedConfiguration$0(SecretsResolver.java:49)
trino at io.airlift.configuration.secrets.SecretsResolver.lambda$getResolvedConfiguration$1(SecretsResolver.java:61)
trino at java.base/java.util.Map.forEach(Unknown Source)
trino at io.airlift.configuration.secrets.SecretsResolver.getResolvedConfiguration(SecretsResolver.java:56)
trino at io.airlift.configuration.secrets.SecretsResolver.getResolvedConfiguration(SecretsResolver.java:48)
trino at io.trino.connector.DefaultCatalogFactory.createCatalog(DefaultCatalogFactory.java:127)
trino at io.trino.connector.LazyCatalogFactory.createCatalog(LazyCatalogFactory.java:44)
trino at io.trino.connector.StaticCatalogManager.lambda$loadInitialCatalogs$1(StaticCatalogManager.java:159)
trino at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
trino at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
trino at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
trino at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30)
trino at java.base/java.util.concurrent.ExecutorCompletionService.submit(Unknown Source)
trino at io.trino.util.Executors.executeUntilFailure(Executors.java:46)
trino at io.trino.connector.StaticCatalogManager.loadInitialCatalogs(StaticCatalogManager.java:153)
trino at io.trino.server.Server.doStart(Server.java:140)
trino at io.trino.server.Server.lambda$start$0(Server.java:79)
trino at io.trino.$gen.Trino_479_stackable0_0_0_dev____20260302_093346_1.run(Unknown Source)
trino at io.trino.server.Server.start(Server.java:79)
trino at io.trino.server.TrinoServer.main(TrinoServer.java:37)
The failure comes from the fact that the pod spec is missing the following statements from the args field:
export CATALOG_ICEBERG_S3_AWS_ACCESS_KEY="$(cat /stackable/secrets/s3-credentials-class/accessKey)"
export CATALOG_ICEBERG_S3_AWS_SECRET_KEY="$(cat /stackable/secrets/s3-credentials-class/secretKey)"
These statements are present in the StatefulSet objects trino-coordinator-default and trino-worker-default.
Since the STS objects are correct, the operator doesn't reconcile them so the pods are left in a failure state.
Forcing the operator pod to restart "fixes" the problem.
More tests
| Result | Kubernetes Version | Notes |
|---|---|---|
| ❌ | 1.35.0 | CI |
| ❌ | 1.35.1 | Local minikube cluster |
| ✅ | 1.35.1 (with MaxUnavailableStatefulSet feature gate disabled)[^1] |
Local minikube cluster |
| ✅ | 1.34.3 | CI |
Attempts to fix
- ❌ Raise the
spec.minReadySecondsfor theStatefulSetfrom0(default) to1seconds.
There is also a bug report related to this field which looks like it may also have implications here. - ❌ Change
spec.updateStrategy.typefor theStatefulSettoOnDelete. - ❌ Do not create
poddisruptionbudgets. - ❌ Uninstall the
commons,secretandlisteneroperator to make sure no annotations are added to the sts/pods.
[^1]: To test with the disabled feature gate, I created a new cluster with the command below:
minikube start \
--kubernetes-version=v1.35.1 \
--nodes 1
--cpus=6
--memory=33518MB
--insecure-registry oci.stackable.tech
--addons metrics-server
--extra-config=controller-manager.feature-gates=MaxUnavailableStatefulSet=false
--extra-config=apiserver.feature-gates=MaxUnavailableStatefulSet=false
- Lingua principale
- Rust
- Stelle
- 63
- Fork
- 13
- Merge medio
- 1g 6h
- PR unite (30g)
- 10
Preparare l'ambiente
- Nessun Dockerfile né file Docker Compose
- Ha un modello di pull request
- Nessuna guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di stackabletech/trino-operator
-
customer-request
Difficoltà 2/5 1-3 ore Idoneità per principianti 60/100
stackabletech/trino-operator#499 ·
I maintainer di solito rispondono entro 1 giorno
-
type/bug
Difficoltà 4/5 3-5 giorni Idoneità per principianti 45/100
stackabletech/trino-operator#936 · 3 commenti ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 3/5 1-2 giorni Idoneità per principianti 45/100
stackabletech/trino-operator#849 ·
I maintainer di solito rispondono entro 1 giorno
-
customer-request type/feature-improvement
Difficoltà 3/5 1-2 giorni Idoneità per principianti 35/100
stackabletech/trino-operator#813 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 5/5 Più di una settimana Idoneità per principianti 25/100
stackabletech/trino-operator#806 ·
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di stackabletech/trino-operator
Issue simili
-
type/bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
stackabletech/kafka-operator#1033 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
bug good first issue needs testing
Difficoltà 2/5 1-3 ore Idoneità per principianti 84/100
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
I maintainer di solito rispondono entro 3 giorni
-
docs: release notes v3.7.0–v3.8.0 footer links to README/CHANGELOG are broken after docs reorgAperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 90/100
farion1231/cc-switch#7744 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
datafusion
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100
apache/iceberg-rust#3297 ·
I maintainer di solito rispondono entro 1 giorno