apache/airflow

Allow passing config_path as a parameter to KubernetesExecutor

Open

#46.293 aberto em 30 de jan. de 2025

Ver no GitHub
 (7 comments) (0 reactions) (0 assignees)Python (16.781 forks)batch import
good first issuekind:featureprovider:cncf-kubernetes

Métricas do repositório

Stars
 (44.809 stars)
Métricas de merge de PR
 (Mesclagem média 7d 18h) (834 fundiu PRs em 30d)

Description

Description

Currently, when using executor_config with KubernetesExecutor, the config_file path must be hardcoded. If we try to pass it dynamically using a Jinja template ("{{ params.config_path }}"), it results in an error.

Example of a working static configuration:

executor_config={"KubernetesExecutor": {"config_file": "/opt/airflow/config/kubernetes/docker-desktop-config.yaml"}}

However, if we attempt to pass it dynamically, it fails:

executor_config={"KubernetesExecutor": {"config_file": "{{ params.config_path }}"}}

This would be useful when deploying DAGs across different environments (e.g., multiple regions) without hardcoding paths.

Proposed Solution:

Modify executor_config to allow templating in KubernetesExecutor.config_file, similar to other templated parameters in Airflow.

Use case/motivation

This would be useful when deploying DAGs across different environments (e.g., multiple regions) without hardcoding paths. Airflow should allow config_file to accept templated values, resolving them at runtime. This feature would improve flexibility when running KubernetesExecutor tasks that need different configurations based on dynamic parameters. Would love to hear thoughts from the community on this! Thanks! 😊

Related issues

This feature would improve Airflow’s flexibility in hybrid execution environments and align with the goals of AIP-61. Would love to hear thoughts from the community on this! 🚀

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Guia do colaborador