apache/airflow

Include debug information in more dagbag warnings

Open

#33 853 ouverte le 28 août 2023

Voir sur GitHub
 (5 commentaires) (0 réactions) (0 assignés)Python (16 781 forks)batch import
good first issuekind:feature

Métriques du dépôt

Stars
 (44 809 stars)
Métriques de merge PR
 (Merge moyen 7j 18h) (834 PRs mergées en 30 j)

Description

Description

Add dag/file information to RemovedInAirflow3Warning, UserWarning, and AirflowProviderDeprecationWarning, so that a user can tell where to go to fix the issue.

Use case/motivation

Some of the RemovedInAirflow3Warnings and UserWarnings aren't actionable because i have no idea what dag they are referring to, such as:

  • RemovedInAirflow3Warning: The use of non-json-serializable params is deprecated and will be removed in a future release
  • AirflowProviderDeprecationWarning: This module is deprecated. Please use airflow.providers.cncf.kubernetes.operators.pod instead. (this one i could take an educated guess and search for pod imports, but many users wouldn't)

Some, like

  • UserWarning: The parameter waiter_countdown has been deprecated to standardize naming conventions. Please use waiter_max_attempts instead. In the future this will default to None and defer to the waiter's default value.
  • RemovedInAirflow3Warning: none_failed_or_skipped Trigger Rule is deprecated. Please use none_failed_min_one_success.

are ok because i can at least search our codebase for those keywords, but they would still be more fixable with debug info.

But even of those, its hard to find at times: for instance, when getting RemovedInAirflow3Warning: airflow.operators.python.task is deprecated. Please use the following instead there is no string i can search for without using regexes or using an IDE to find uses of the function.

Unfortunately i do know this would likely be a case-by-case fix.

Related issues

https://github.com/apache/airflow/issues/14613

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Guide contributeur