apache/airflow

Missing docs for logging

Open

#10.593 geöffnet am 27. Aug. 2020

Auf GitHub ansehen
 (5 Kommentare) (14 Reaktionen) (0 zugewiesene Personen)Python (16.781 Forks)batch import
area:loggingcontributors-workshopgood first issuekind:documentation

Repository-Metriken

Stars
 (44.809 Stars)
PR-Merge-Metriken
 (Durchschn. Merge 7T 18h) (834 gemergte PRs in 30 T)

Beschreibung

Hello,

The configuration of the Airflow logger is not easy, as in most applications. We have many loggers, some loggers generate multiple streams. This is very confusing for end-users, and the lack of documentation doesn't make it any easier.

We have following loggers

  • root logger
  • airflow.processor
  • airflow.task
  • flask_appbuilder
  • airflow.processor_manager

airflow.task creates a new log file for each pair of task instances and try numbers. These entries are accessible via the Web UI. airflow.processor creates a new log file for each DAG File. flask_appbuilder is only used to filter logs records because the FAB is too verbose, so in most cases, we don't have to configure it specially.

Webserver writes logs to root logger. Worker writes logs to airflow.processor and root logger Scheduler writes logs to airflow.processor, airflow.processor_manager and root logger

I would be happy if we had a description of the default configuration and the methods on how to influence these configurations were described. Currently, the documentation only describes configurations for airflow.task.

Best regards, Kamil Breguła

Contributor Guide