Unable to specify ssl custom CA for operator pod using KubernetesPodOperator · apache/airflow#53192

(3 comments) (0 reactions) (0 assignees)Python (44,809 stars) (16,781 forks)batch import

area:providersgood first issuekind:bugprovider:cncf-kubernetes

Description

Apache Airflow version

Other Airflow 2 version (please specify below)

If "Other Airflow 2 version" selected, which one?

2.10.5

What happened?

We are running Airflow in our own Kubernetes cluster, which uses an internal intermediate Certificate Authority (CA) for service-to-service communication.

Due to a known limitation of urllib3 (used by Python), which does not handle intermediate root certificates properly, we explicitly provide the full certificate chain for Airflow and other components. In most cases, this is resolved by passing the complete certificate bundle via an environment variable, and pointing relevant Airflow components to it using standard ENV configuration. This works correctly outside the Kubernetes executor.

However, when using the KubernetesPodOperator with Kubernetes Executor, the overridden certificate (provided via ENV) is ignored. This results in an SSL validation error during pod spawn. We could not find any documented way — either in the official docs or through experimentation — to explicitly pass this custom CA to the pods launched by the KubernetesPodOperator spawner pod.

What you think should happen instead?

There should be a supported and documented method to pass additional CA certificates (e.g. via ENV or config) to the operator pods spawned by the Kubernetes Executor, similar to how it works for other Airflow components.

How to reproduce

The pods launched by the Kubernetes Executor do not inherit or respect the custom CA provided via ENV, leading to SSL errors during execution.

Configure Airflow to use a custom CA bundle via ENV.

Use the Kubernetes Executor with KubernetesPodOperator.

Observe that spawned pods fail with SSL-related errors due to untrusted intermediate CA.

Operating System

k8s 1.30.10

Versions of Apache Airflow Providers

No response

Deployment

Official Apache Airflow Helm Chart

Deployment details

Environment: Airflow version: 2.10.5 K8s: 1.30.10 Certs: custom fullchain Executor: Kubernetes Executor Operator: KubernetesPodOperator

Anything else?

Error:

urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='10.233.0.1', port=443): Max retries exceeded with url: /api/v1/namespaces/airflow/pods?labelSelector=dag_id%3Dcifar100_classification%2Ckubernetes_pod_operator%3DTrue%2Crun_id%3Dmanual__2025-07-10T132839.5223690000-4c7fe4717%2Ctask_id%3Dtrain_model%2Calready_checked%21%3DTrue%2C%21airflow-worker (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get issuer certificate (_ssl.c:1010)')))

Are you willing to submit PR?

Yes I am willing to submit a PR!

Code of Conduct

I agree to follow this project's Code of Conduct

Contributor guide

Tech stack: pythondockerkubernetes
Domain: backenddevopsinfrastructure
Issue type: bug
Difficulty: 4
Estimated time: 3-5 days
Activity status: fresh
Clarity: mostly clear
Prerequisites: Kubernetes basicsSSL/TLS understandingAirflow Kubernetes Executor internals
Newbie friendliness: 20
Research direction: Investigate how the KubernetesPodOperator spawns pods, particularly how environment variables (like SSL CERT FILE) are passed. Look at the Kubernetes executor source code (e.g., airflow/providers/cncf/kubernetes/executors.py and pod generator.py) to understand if custom CA config is lost. Check for existing env or config options in the Helm chart values. Consider testing with a custom CA bundle via env and debug SSL errors.