Tasks that are running when a DAG run reaches its timeout should optionally support `on_killed` · apache/airflow#41036

(9 评论) (1 反应) (0 负责人)Python (16,781 fork)batch import

area:Schedulergood first issuekind:feature

仓库指标

Star: (44,809 star)
PR 合并指标: (平均合并 7天 18小时) (30 天内合并 834 个 PR)

描述

Description

The current functionality for task instances that are in the running state when a DAG run reaches its timeout is controversial (see #30264). It'd be great if users could optionally allow tasks' on_kill method to be called at this time.

Use case/motivation

Some users would like for externally running workloads (e.g. snowflake, emr, bigquery, etc etc) to stop executing when a DAG run times out.

Related issues

#30264

Are you willing to submit a PR?

Yes I am willing to submit a PR!

Code of Conduct

I agree to follow this project's Code of Conduct

贡献者指南

研究方向: 检查 Airflow 如何处理 DAG 运行超时和任务终止，重点关注 `on kill` 回调。研究 `taskinstance.py` 和 `dagrun.py` 中的现有逻辑，了解任务在何处被终止，以及如何在超时期间可选地调用 `on kill`。考虑添加一个配置标志。
技术栈: python
领域: backend
议题类型: 功能
难度: 2
预计时间: 1-2 天
活动状态: 活跃
清晰度: 清晰
前置要求: PythonAirflow internals
新手友好度: 60