Fix handling AirflowFailException `on_retry_callback`
#60 172 ouverte le 6 janv. 2026
Métriques du dépôt
- Stars
- (44 809 stars)
- Métriques de merge PR
- (Merge moyen 7j 18h) (834 PRs mergées en 30 j)
Description
Description
This was previously possible in Airflow 2 using this solution. However, in Airflow 3 RuntimeTaskInstanceProtocol does not expose a method for modifying the task state directly.
My suggestion would be to create a new exception type that can be raised from on_retry_callback to request a state change to either success or failure.
Update: While this is not specified, it seema a bug that it does not work that way: AirflowFailException raised in on_retry_callback or any other state-change callback should immediately fail the task.
Use case/motivation
We have some long-running tasks (>24hr) that sometimes fail fast due to transient issues. We want to be able to retry whenever these fast fails occur. However, if a failure occurs hours into the task, we would prefer not to retry, as the task must be completed by a certain time and failures of this sort typically represent an actual issue we want to look into.
Related issues
Just the discussion item here from years ago as far as I can tell.
Are you willing to submit a PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct