Bug with the advanced logging configuration documentation with remote logging and custom logging config class for airflow version 3.2.2

Compare the Airflow 3.2.2 advanced logging configuration page linked in the issue with the updated 3.3.1 documentation, focusing on custom logging_config_class with remote logging. Document the required REMOTE_TASK_LOG and DEFAULT_REMOTE_CONN_ID configuration, and confirm that the 3.2.2 page explains how Azure remote logs become available.

Under which category would you file this issue?

Documentation

Apache Airflow version

3.2.2

What happened and how to reproduce it?

I have got an Airflow self hosted deployment on Azure Kubernetes cluster with remote logging enabled to azure blob storage. I was migrating my existing deployment from airflow version 2.8.4 to 3.2.2.

My setup involves remote_logging enabled with logs copied to azure blob storage. I use a custom logging_config_class. This was all working on airflow version 2.8.4.

I followed what was mentioned in the documentation of version 3.2.2 advanced logging configuration.

After migrating to v3.2.2 I was getting this warning in the scheduler logs

warning ] remote_log_handler_unavailable [airflow.logging.remote] loc=log.py:236 note='Remote log handler could not be loaded; logs will be available locally only.'

And the logs were not copied to my azure blob storage container.

Upon going through the source code for the remote logging to work, I found that we need to set the following global variables in the python module.

REMOTE_TASK_LOG: RemoteLogIO | None = MyRemoteLogIO()
DEFAULT_REMOTE_CONN_ID: str | None = "my_remote_conn"

This wasn't captured in the airflow 3.2.2 documentation and after fixing this, the scheduler warning related to remote logging disappeared and the remote logs were written to azure blob storage.

What you think should happen instead?

The documentation for the advanced logging that uses custom logging_config_class along with remote logging enabled scenario is not captured correctly.

The documentation for airflow version 3.3.1 is updated with this information, but the documentation for airflow version 3.2.2 should be updated with this information.

Operating System

NAME="Debian GNU/Linux", VERSION_ID="12", VERSION_CODENAME=bookworm

Deployment

Official Apache Airflow Helm Chart

Apache Airflow Provider(s)

No response

Versions of Apache Airflow Providers

apache-airflow-providers-amazon          9.29.0
apache-airflow-providers-apache-livy     4.5.6
apache-airflow-providers-cncf-kubernetes 10.17.1
apache-airflow-providers-common-compat   1.15.0
apache-airflow-providers-common-io       1.7.2
apache-airflow-providers-common-sql      2.0.0
apache-airflow-providers-fab             3.6.4
apache-airflow-providers-http            6.0.2
apache-airflow-providers-microsoft-azure 13.3.0
apache-airflow-providers-postgres        6.7.0
apache-airflow-providers-smtp            3.0.1
apache-airflow-providers-ssh             5.0.2
apache-airflow-providers-standard        1.13.1

Official Helm Chart version

1.22.0 (latest released)

Kubernetes Version

1.35.3

Helm Chart configuration

Airflow logging related YAML configuration

        logging:
            remote_logging: "True"
            remote_log_conn_id: "azblob_logging"
            remote_base_log_folder: "wasb://wasb-airflow-logs-aks"
            base_log_folder: "/opt/airflow/logs"
            logging_level: "INFO"
            fab_logging_level: "WARN"
            colored_console_log: "True"
            colored_formatter_class: "airflow.utils.log.colored_log.CustomTTYColoredFormatter"
            simple_log_format: "%%(asctime)s %%(levelname)s - %%(message)s"
            dag_processor_child_process_log_directory: "/opt/airflow/logs/dag_processor"
            task_log_reader: "task"
            encrypt_s3_logs: "False"
            delete_local_logs: "False"
            logging_config_class: "config.log_config.LOGGING_CONFIG"

        azure_remote_logging:
            remote_wasb_log_container: "airflow-logs"

Docker Image customizations

Not Applicable

Anything else?

This was my original logging configuration

from copy import deepcopy

from airflow.config_templates.airflow_local_settings import DEFAULT_LOGGING_CONFIG

LOGGING_CONFIG = deepcopy(DEFAULT_LOGGING_CONFIG)

# Prevent task log records from propagating to the root console handler.
# The default propagate=True means every record is handled twice (FileTaskHandler
# + root console), which the Airflow UI merges into duplicate log lines.
LOGGING_CONFIG["loggers"]["airflow.task"]["propagate"] = False

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct