apache/airflow

[Helm Chart] Support multiple DagProcessors

Open

#26,494 opened on Sep 19, 2022

View on GitHub
 (11 comments) (2 reactions) (0 assignees)Python (16,781 forks)batch import
area:helm-chartgood first issuekind:feature

Repository metrics

Stars
 (44,809 stars)
PR merge metrics
 (Avg merge 7d 18h) (834 merged PRs in 30d)

Description

Description

Airflow 2.4.0 has introduced support for Support multiple DagProcessors parsing files from different locations.

However, Helm chart only supports Provision Standalone Dag Processor (or at least will support in Airflow Helm Chart 1.7.0).

Use case/motivation

For Airflow Helm Chart users take advantage of these new AIP-43 features, it is necessary to add support for provision multiple DagProcessors in Helm chart.

Related issues

Helmchart: Provision Standalone Dag Processor Support multiple DagProcessors parsing files from different locations

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Contributor guide

Research direction
Examine the current Helm chart's dagProcessor deployment and the upstream changes in Airflow 2.4.0 for multiple DagProcessors, then design a Helm values schema to support multiple instances.
Tech stack
pythonkubernetes
Domain
infrastructuredevops
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
HelmKubernetesGit
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
45