ExternalTaskSensor: An option to use the data_interval variables rather than execution_date
#26,429 opened on 2022年9月16日
Repository metrics
- Stars
- (44,809 stars)
- PR merge metrics
- (平均マージ 7d 18h) (30d で 834 merged PRs)
説明
Description
Since execution_date is deprecated, it makes sense that this sensor or a new version of it (for backward compatibility) uses the data_interval variables.
Use case/motivation
I want to make sure that the data interval this DAG is about to operate on is covered by another DAG. For example, data was loaded up to this DAG's data_interval_end.
I believe the sensor itself is simple to write. It just uses data_interval_end rather than execution_date and it doesn't look for one specific task, any external.data_interval_end > self.data_interval_end is acceptable.
This is the equivalent of ExternalTaskSensor with execution_delta set to self.interval_start - closest_external_interval.interval_start which can be calculated manually but will fail later if the external DAG schedule is changed.
Related issues
No response
Are you willing to submit a PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct