apache/airflow

Helm chart: Git-sync v4 incorrect mix of branch, ref, rev

Open

#42 918 ouverte le 10 oct. 2024

Voir sur GitHub
 (3 commentaires) (0 réactions) (0 assignés)Python (16 781 forks)batch import
area:helm-chartgood first issuekind:bug

Métriques du dépôt

Stars
 (44 809 stars)
Métriques de merge PR
 (Merge moyen 7j 18h) (834 PRs mergées en 30 j)

Description

Official Helm Chart version

1.15.0 (latest released)

Apache Airflow version

2.5.3

Kubernetes Version

1.30.4-gke.1348000

Helm Chart configuration

dags:
  gitSync:
    branch: main
    rev: HEAD
    ref: v1.2.3
    repo: git@github.com:<ORG>/<REPO>.git
    depth: 1
    enabled: true
    sshKeySecret: <my-key>
    subPath: src/<MY>/<PATH>
    knownHosts: |
      github.com ssh-rsa <KEY>

Docker Image customizations

No response

What happened

I pushed code to the main branch, and this was picked up by the gitSync sidecar and it updated the Airflow Dags to use the new code

What you think should happen instead

It should not have updated the code, because the ref is "frozen" to version v1.2.3. This is a production environment, and that's the whole reason I froze it by pointing it at a Git tag - so that pushing new code won't automatically deploy until it is reviewed/tagged/released.

How to reproduce

  1. Have your git repo have a branch named main, and a tag named v1.2.3 that points to main.
  2. Set this in your values.yaml
images:
 gitSync:
   repository: registry.k8s.io/git-sync/git-sync
   tag: v4.1.0
dags:
 gitSync:
   branch: main
   rev: HEAD
   ref: v1.2.3
   repo: git@github.com:<ORG>/<REPO>.git
   enabled: true
  1. Deploy this
  2. Make a new commit to the main branch, and push it
  3. You will see that the gitSync container updates the code with the new commit, even though it is supposed to be "frozen" to v1.2.3

Anything else

I dug into this, and it's caused by incorrectly mapping 3 parameters to the gitSync container. The latest Helm chart (v1.15) by default uses this image https://github.com/apache/airflow/blob/ff7463b1624d91fb6478ec367bbf84b88fb9b83d/chart/values.yaml#L124-L127 which I believe is from this source code here https://github.com/kubernetes/git-sync/tree/v4.1.0.

The 3 incorrect params are:

  • branch
  • rev
  • ref

In v4.x of gitSync, the params branch and rev are deprecated, and only ref should be used. But if any of the deprecated params are specified, they override ref, which was the root cause of my issue.

These are explicitly marked as deprecated (from v3 to v4) in the migration guide here, and inspecting the source code you can see they are:

I fixed it using a hack, by setting branch and rev to empty strings, but this is not ideal.

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Guide contributeur