apache/airflow

DAG Import Errors message is dangling in web-interface

Open

#45.227 geöffnet am 27. Dez. 2024

Auf GitHub ansehen
 (22 Kommentare) (0 Reaktionen) (0 zugewiesene Personen)Python (16.781 Forks)batch import
area:MetaDBarea:coregood first issuekind:bug

Repository-Metriken

Stars
 (44.809 Stars)
PR-Merge-Metriken
 (Durchschn. Merge 7T 18h) (834 gemergte PRs in 30 T)

Beschreibung

Apache Airflow version

Other Airflow 2 version (please specify below)

If "Other Airflow 2 version" selected, which one?

2.10.2

What happened?

We have this message in the Airflow's web-interface no matter what. This DAG can exist or not - the error still shows.

We have found many different topics across the Internet for various Airflow versions. Some of them from 2018. So users constantly witness these dangling messages, but there are no clarification for this case.

Also we don't have an understanding what is a source for this notification message. It sits in the Airflow DB and webserver is selecting it from there or other core components are generating it for webserver somehow.

What you think should happen instead?

Even though the error message itself is far from perfect, we expect that if DAG doesn't exist anymore, the error message should disappear.

How to reproduce

We use gitsync option for storing DAGs:

# DAGs Config
dags:
  gitSync:
    enabled: true
    repo: "https://my.repo/path/name.git"
    branch: "master"
    rev: "HEAD"
    subPath: "airflowk8s/dags"
    credentialsSecret: my-secret
    period: 5s
    maxFailures: 10

And our DAG is just script.py like that, nothing complicated: print("D0ne!")

So I guess steps should be:

  1. Create a broken DAG in remote repository;
  2. Get the error message in the Airflow's web-interface;
  3. Delete a broken DAG in remote repository;
  4. Wait for your gitsync;
  5. The error message in the Airflow's web-interface will dangle.

But this bug appears randomly, so we don't know exact reason.

Operating System

Kubernetes v1.31.0-eks

Versions of Apache Airflow Providers

No response

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Contributor Guide