apache/airflow

Launch Pod/job in separate namespace and support in helm chart

Open

#25,222 opened on Jul 21, 2022

View on GitHub
 (6 comments) (1 reaction) (0 assignees)Python (16,781 forks)batch import
area:helm-chartgood first issuekind:feature

Repository metrics

Stars
 (44,809 stars)
PR merge metrics
 (Avg merge 7d 18h) (834 merged PRs in 30d)

Description

Description

Our engineers want to run jobs in a separate namespace from the installed release via helm chart.

It looks to me like right now the rbac configuration has two choices

  • multicluster - generates ClusterRole
  • !multicluster - generales rolebinding in RELEASE namespace.

Why these don't work for us

  • multicluster is too wide, we want to restrict the namespace
  • !multicluster - I think only lets stuff be generated in the same namespace? Or will airflow be able to generate pods in another namespace

Sorry for the ignorant questions - don't really know airflow, just trying to help out the engineers.

Use case/motivation

Our engineers want to run jobs in a separate namespace from the installed release via helm chart.

Related issues

There are some about pod override of namespace, I'm not sure due to inexperience with Airflow

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Contributor guide

Research direction
Study the Airflow Helm chart's RBAC configuration, especially the conditions for multicluster vs non multicluster. Identify where namespace references are hardcoded and propose a mechanism to allow users to specify a separate namespace for pod/job execution. Review related issues and pull requests for prior art. Ensure the solution respects security constraints and does not grant excessive permissions.
Tech stack
pythonkubernetesdocker
Domain
devopsinfrastructure
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Mostly clear
Prerequisites
GitKubernetesHelm
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
65