Allow passing config_path as a parameter to KubernetesExecutor
#46.293 aberto em 30 de jan. de 2025
Métricas do repositório
- Stars
- (44.809 stars)
- Métricas de merge de PR
- (Mesclagem média 7d 18h) (834 fundiu PRs em 30d)
Description
Description
Currently, when using executor_config with KubernetesExecutor, the config_file path must be hardcoded. If we try to pass it dynamically using a Jinja template ("{{ params.config_path }}"), it results in an error.
Example of a working static configuration:
executor_config={"KubernetesExecutor": {"config_file": "/opt/airflow/config/kubernetes/docker-desktop-config.yaml"}}
However, if we attempt to pass it dynamically, it fails:
executor_config={"KubernetesExecutor": {"config_file": "{{ params.config_path }}"}}
This would be useful when deploying DAGs across different environments (e.g., multiple regions) without hardcoding paths.
Proposed Solution:
Modify executor_config to allow templating in KubernetesExecutor.config_file, similar to other templated parameters in Airflow.
Use case/motivation
This would be useful when deploying DAGs across different environments (e.g., multiple regions) without hardcoding paths. Airflow should allow config_file to accept templated values, resolving them at runtime. This feature would improve flexibility when running KubernetesExecutor tasks that need different configurations based on dynamic parameters. Would love to hear thoughts from the community on this! Thanks! 😊
Related issues
This feature would improve Airflow’s flexibility in hybrid execution environments and align with the goals of AIP-61. Would love to hear thoughts from the community on this! 🚀
Are you willing to submit a PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct