GKEStartJobOperator impersonation chain not working with Kubernetes RBAC

Apache Airflow version

Other Airflow 2 version (please specify below)

If "Other Airflow 2 version" selected, which one?

2.10.5

What happened?

I am trying to run the GKEStartJobOperator using impersonation chain from airflow. I have setup RBAC in the gke cluster to give that iam account permissions using the method shown here https://cloud.google.com/kubernetes-engine/docs/how-to/role-based-access-control#rolebinding. However, when running GKEStartJobOperator it seems to call the GKE cluster with the unqiue id of the service account in the impersonation chain rather than the email. This causes the RBAC to fail as it cant match the User. It works if i put the unqiue id directly as a user subject.

What you think should happen instead?

The follow error happens User \”<unique_id>\” cannot create resource \"jobs\" in API group \"batch\" in the namespace \"airflow-jobs\": requires one of [\"container.jobs.create\"] permission(s). I expected it to use the email in the impersonation chain rather than the unique id.

How to reproduce

1. set up rbac as follows

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: airflow-jobs
  name: job-creator
rules:
  - apiGroups: [ "batch" ]
    resources: [ "jobs", ]
    verbs: [ "create" , "list", "get", "update", "patch", "delete"]
  - apiGroups: [ "batch" ]
    resources: [ "jobs/status" ]
    verbs: [ "get" ]
  - apiGroups: [ "" ]
    resources: [ "pods" ]
    verbs: [ "list", "get"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: job-creator-binding
  namespace: airflow-jobs
subjects:
  - kind: User
    name: <iam-service-account>
    namespace: airflow-jobs
roleRef:
  kind: Role
  name: job-creator
  apiGroup: rbac.authorization.k8s.io

2. create a dag with operator configure with impersonation chain

job_task = GKEStartJobOperator(
        task_id="job_task",
        project_id=“<project-id>,
        location="europe-west1",
        cluster_name=“<gke-cluster-name>“,
        namespace="airflow-jobs",
        image="perl:5.34.0",
        cmds=["perl", "-Mbignum=bpi", "-wle", "print bpi(2000)"],
        name="test-job",
        ttl_seconds_after_finished=60,
        wait_until_job_complete=True,
        deferrable=True,
        impersonation_chain=“<iam service account>”,
    )

Operating System

linux

Versions of Apache Airflow Providers

apache-airflow==2.10.5+composer apache-airflow-providers-apache-beam==6.0.4 apache-airflow-providers-celery==3.10.0 apache-airflow-providers-cncf-kubernetes==10.1.0 apache-airflow-providers-common-compat==1.6.0 apache-airflow-providers-common-io==1.5.4 apache-airflow-providers-common-sql==1.26.0 apache-airflow-providers-dbt-cloud==4.3.3 apache-airflow-providers-fab==1.5.3 apache-airflow-providers-ftp==3.12.3 apache-airflow-providers-google==15.1.0 apache-airflow-providers-hashicorp==4.1.1 apache-airflow-providers-http==5.2.2 apache-airflow-providers-imap==3.8.3 apache-airflow-providers-mysql==6.2.2 apache-airflow-providers-openlineage==2.2.0 apache-airflow-providers-postgres==6.1.3 apache-airflow-providers-redis==4.0.2 apache-airflow-providers-sendgrid==4.0.1 apache-airflow-providers-smtp==2.0.3 apache-airflow-providers-sqlite==4.0.2 apache-airflow-providers-ssh==4.0.1 apache-airflow-providers-standard==1.0.0

Deployment

Google Cloud Composer

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct