apache/airflow

Include debug information in more dagbag warnings

Open

#33.853 aperta il 28 ago 2023

Vedi su GitHub
 (5 commenti) (0 reazioni) (0 assegnatari)Python (16.781 fork)batch import
good first issuekind:feature

Metriche repository

Star
 (44.809 star)
Metriche merge PR
 (Merge medio 7g 18h) (834 PR mergiate in 30 g)

Descrizione

Description

Add dag/file information to RemovedInAirflow3Warning, UserWarning, and AirflowProviderDeprecationWarning, so that a user can tell where to go to fix the issue.

Use case/motivation

Some of the RemovedInAirflow3Warnings and UserWarnings aren't actionable because i have no idea what dag they are referring to, such as:

  • RemovedInAirflow3Warning: The use of non-json-serializable params is deprecated and will be removed in a future release
  • AirflowProviderDeprecationWarning: This module is deprecated. Please use airflow.providers.cncf.kubernetes.operators.pod instead. (this one i could take an educated guess and search for pod imports, but many users wouldn't)

Some, like

  • UserWarning: The parameter waiter_countdown has been deprecated to standardize naming conventions. Please use waiter_max_attempts instead. In the future this will default to None and defer to the waiter's default value.
  • RemovedInAirflow3Warning: none_failed_or_skipped Trigger Rule is deprecated. Please use none_failed_min_one_success.

are ok because i can at least search our codebase for those keywords, but they would still be more fixable with debug info.

But even of those, its hard to find at times: for instance, when getting RemovedInAirflow3Warning: airflow.operators.python.task is deprecated. Please use the following instead there is no string i can search for without using regexes or using an IDE to find uses of the function.

Unfortunately i do know this would likely be a case-by-case fix.

Related issues

https://github.com/apache/airflow/issues/14613

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Guida contributor