Git-sync with kubernetesExecutors and using https
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- git, kubernetes, rust
- Domain
- devops, infrastructure
Research direction
Compare the git-sync settings shown for the webserver and scheduler with those generated for kubernetesExecutors, focusing on propagation of gitSyncConf and the one-time flag. Review the discussion in PR #456, especially the linked issue comment, then verify that all three workload types produce equivalent git-sync configuration for HTTPS repositories.
Written by the indexing model from the issue text.
Description
Affected Stackable version
Nightly
Affected Apache Airflow version
2.8.1
Current and expected behavior
Using this AirflowCluster:
---
apiVersion: airflow.stackable.tech/v1alpha1
kind: AirflowCluster
metadata:
name: airflow
spec:
image:
productVersion: 2.8.1
clusterConfig:
loadExamples: false
exposeConfig: false
listenerClass: external-unstable
credentialsSecret: simple-airflow-credentials
dagsGitSync:
- repo: https://custom.gitlab.com/repos/airflow-dags.git
branch: "main"
gitFolder: "dags"
depth: 10
wait: 20
credentialsSecret: gitcredentials
gitSyncConf:
--rev: HEAD
--git-config: http.sslCAInfo:/tmp/ca-cert/ca.crt
volumeMounts:
- name: "cert-ca"
mountPath: "/tmp/ca-cert"
volumes:
- name: "cert-ca"
configMap:
name: "cert-ca"
items:
- key: "ca.crt"
path: "ca.crt"
webservers:
roleGroups:
default:
replicas: 1
podOverrides:
spec:
containers:
- name: gitsync-1
volumeMounts:
- name: "cert-ca"
mountPath: "/tmp/ca-cert"
kubernetesExecutors:
config:
resources:
cpu:
min: 400m
max: 800m
memory:
limit: 2Gi
podOverrides:
spec:
containers:
- name: gitsync-1
volumeMounts:
- name: "cert-ca"
mountPath: "/tmp/ca-cert"
schedulers:
roleGroups:
default:
replicas: 1
podOverrides:
spec:
containers:
- name: gitsync-1
volumeMounts:
- name: "cert-ca"
mountPath: "/tmp/ca-cert"
The gitSyncConf does not seem to get propagated for the kubernetesExecutors. Also the git-sync container has a one-time=true flag set for the git-sync command
Follow up this MR: https://github.com/stackabletech/airflow-operator/pull/456#issuecomment-2230116710
Expected behavior: the webserver, scheduler and kubernetesExecutors are configured exactly the same for the git-sync containers
Possible solution
No response
Additional context
No response
Environment
No response
Would you like to work on fixing this bug?
None
- Dominant language
- Rust
- Stars
- 32
- Forks
- 6
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 12
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from stackabletech/airflow-operator
-
Difficulty 1/5 Under an hour Newbie friendliness 82/100
stackabletech/airflow-operator#898 ·
Maintainers usually reply within 1 day
-
good first issue type/feature-improvement
Difficulty 4/5 3-5 days Newbie friendliness 45/100
stackabletech/airflow-operator#760 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 42/100
stackabletech/airflow-operator#697 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
stackabletech/airflow-operator#638 ·
Maintainers usually reply within 1 day
-
type/feature-new
Difficulty 5/5 Over a week Newbie friendliness 25/100
stackabletech/airflow-operator#593 ·
Maintainers usually reply within 1 day
All issues in stackabletech/airflow-operator
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
2 advisory ignore entries no longer match anything (`bytes`, `time` already patched in `Cargo.lock`)Possibly taken @zyl71 claimed this today. Open
Difficulty 1/5 Under an hour Newbie friendliness 88/100
-
Difficulty 1/5 Under an hour Newbie friendliness 85/100
stackabletech/druid-operator#905 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
bytecodealliance/wrpc#1785 ·
Maintainers usually reply within 1 day
-
2 advisory ignore entries no longer match anything (`quick-xml` already patched in `Cargo.lock`)Open
Difficulty 1/5 Under an hour Newbie friendliness 85/100
Maintainers usually reply within 1 day