Log exceptions as warnings till the final attempt · apache/airflow#32246

(10 评论) (1 反应) (0 负责人)Python (16,781 fork)batch import

good first issuekind:feature

仓库指标

Star: (44,809 star)
PR 合并指标: (平均合并 7天 18小时) (30 天内合并 834 个 PR)

描述

Description

We have an integration with sentry on Google cloud composer. We would like to be alerted for any exceptions happening in airflow , but don't want to be alerted when those exceptions would be retried. We tried setting alert conditions on number of errors, but it does not solve the problem as we have hundreds of instances of a DAG running.

Requirements:

Log as warning when try_number is less than equal to max_tries.
Log as an error in the final try.

Use case/motivation

No response

Related issues

No response

Are you willing to submit a PR?

Yes I am willing to submit a PR!

Code of Conduct

I agree to follow this project's Code of Conduct

贡献者指南

研究方向: 研究 Airflow 中任务实例的执行流程，特别是 `airflow/models/taskinstance.py` 中的日志记录机制。查看 `try number` 和 `max tries` 是如何被跟踪的，以及异常在何处被记录。此更改可能涉及根据当前尝试是否为最终重试来修改日志级别。检查 ` execute task` 或类似方法中现有的日志调用，并考虑对非最终尝试使用 `logging.Logger.warning`，对最终尝试使用 `logging.Logger.error`。
技术栈: python
领域: backend
议题类型: 功能
难度: 3
预计时间: 1-2 天
活动状态: 活跃
清晰度: 清晰
前置要求: PythonAirflow
新手友好度: 70