Investigate possible race conditions when gitsync-ing DAG submodules

Open
#417 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
git, rust
Domain
devops

Research direction

Start by examining the gitsync behavior around DAG submodules. Reproduce syncs while a DAG is running and add a new submodule, then verify that the running DAG remains consistent and that the new submodule's DAGs are loaded and registered.

Written by the indexing model from the issue text.

Description

Description

As a user of Airflow I want to be sure that DAGs that use submodules are seamlessly updated and registered with Airflow via gitsync. This covers the following:

### Tasks
- [ ] Check that DAGs are consistent i.e. that the state when the DAG is started is the same despite subsequent syncs while the DAG is still running
- [ ] Check that "new" submodules are correctly loaded/registered
Dominant language
Rust
Stars
32
Forks
6
Avg merge
1d 6h
Merged PRs (30d)
12

Getting set up

  • No Dockerfile or Docker Compose file
  • Has a pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from stackabletech/airflow-operator

All issues in stackabletech/airflow-operator

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.