仓库指标
- Star
- (44,809 star)
- PR 合并指标
- (平均合并 7天 18小时) (30 天内合并 834 个 PR)
描述
Currently in the ExternalTaskSensor there are two ways two set the execution date of a DAG/task the sensor is waiting on:
- Provide the
execution_deltawhich simply adds/subtracts a time value to/from theexecution_dateof the sensor itself - Provide the
execution_date_fnwhich is a function that takes in just theexecution_dateand returns a manipulated one.
I believe that the correct way to go about handling the execution_date for the ExternalTaskSensor should be analogous to the way we handle it for other operators, such as the ExternalTaskMarker, or more relevantly, the TriggerDagRunOperator, which is an operator usually used in conjunction with the ExternalTaskSensor.
Specifically, we should remove the execution_delta parameter and replace it with a templated execution_date parameter. This not only covers the existing execution_delta use case through e.g. {{ execution_date - macros.timedelta(days=1) }}, but also provides more flexibility by enabling the benefits of templated parameters in general, such as pulling from xcoms, reading from DAG/operator params, using macros, etc. It also establishes the pattern that we see in the previously mentioned operators.
The proposed solution is also sufficient enough to remove the execution_date_fn because we can register custom macros to manipulate the execution date the same way as an execution_date_fn would. But I do realize we may want to keep it for backwards compatibility reasons.
First time contributor here so happy to implement/help with this to get some airflow OSS chops.