apache/airflow

Last automated data interval is always available in custom timetable

Geschlossen

#27.672 geöffnet am 14.11.2022

 (6 Kommentare) (0 Reaktionen) (0 zugewiesene Personen)Python (16.781 Forks)batch import
area:coregood first issuekind:documentation

Repository-Metriken

Stars
 (44.809 Sterne)
PR-Merge-Metriken
 (Durchschn. Merge 7T 18h) (834 gemergte PRs in 30 T)

Beschreibung

Apache Airflow version

Other Airflow 2 version (please specify below)

What happened

I'm writing an example custom timetable. Implemented next_dagrun_info. From the docs and examples the parameter last_automated_data_interval should be None if there are no previous runs.

However, when I start up the example:

  1. I can confirm that the table dag_run is empty.
  2. when starting (unpausing the DAG) for the first time, the last_automated_data_interval is a data interval and not None as specified by documentation.

This raises the question of how to determine the first DAG run (probably could subtract the DataInterval start and start_date from the DAG (if possible).

Here is an example from the logs: airflow-feat-scheduler | [2022-11-14 19:57:58,934] {WorkDayTimetable.py:28} INFO - last_automated_data_interval: DataInterval(start=DateTime(2022, 11, 10, 0, 0, 0, tzinfo=Timezone('UTC')), end=DateTime(2022, 11, 11, 0, 0, 0, tzinfo=Timezone('UTC')))

I'm using Airflow 2.4.2.

What you think should happen instead

The value of the parameter should be None as specified in the docs.

How to reproduce

Should be reproducible by running the example given in the docs and logging the value of the parameter: last_automated_data_interval. Should appear in the logs.

Operating System

macOs Ventura

Versions of Apache Airflow Providers

No response

Deployment

Docker-Compose

Deployment details

No response

Anything else

The problem occurs every time.

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Contributor Guide