Tasks that are running when a DAG run reaches its timeout should optionally support `on_killed` · apache/airflow#41036

(9 留言) (1 反應) (0 負責人)Python (16,781 fork)batch import

area:Schedulergood first issuekind:feature

倉庫指標

Star: (44,809 star)
PR 合併指標: (平均合併 7天 18小時) (30 天內合併 834 個 PR)

描述

Description

The current functionality for task instances that are in the running state when a DAG run reaches its timeout is controversial (see #30264). It'd be great if users could optionally allow tasks' on_kill method to be called at this time.

Use case/motivation

Some users would like for externally running workloads (e.g. snowflake, emr, bigquery, etc etc) to stop executing when a DAG run times out.

Related issues

#30264

Are you willing to submit a PR?

Yes I am willing to submit a PR!

Code of Conduct

I agree to follow this project's Code of Conduct

貢獻者指南

研究方向: 檢查 Airflow 如何處理 DAG 運行超時和任務終止，重點關注 `on kill` 回調。研究 `taskinstance.py` 和 `dagrun.py` 中的現有邏輯，了解任務在何處被終止，以及如何在超時期間可選地調用 `on kill`。考慮添加一個配置標誌。
技術棧: python
領域: backend
議題類型: 功能
難度: 2
預計時間: 1-2 天
活動狀態: 活躍
清晰度: 清晰
前置要求: PythonAirflow internals
新手友善度: 60