ExternalTaskSensor: An option to use the data_interval variables rather than execution_date
#26,429 创建于 2022年9月16日
仓库指标
- 星标
- (44,809 个星标)
- PR 合并指标
- (PR 指标待抓取)
描述
Description
Since execution_date is deprecated, it makes sense that this sensor or a new version of it (for backward compatibility) uses the data_interval variables.
Use case/motivation
I want to make sure that the data interval this DAG is about to operate on is covered by another DAG. For example, data was loaded up to this DAG's data_interval_end.
I believe the sensor itself is simple to write. It just uses data_interval_end rather than execution_date and it doesn't look for one specific task, any external.data_interval_end > self.data_interval_end is acceptable.
This is the equivalent of ExternalTaskSensor with execution_delta set to self.interval_start - closest_external_interval.interval_start which can be calculated manually but will fail later if the external DAG schedule is changed.
Related issues
No response
Are you willing to submit a PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct