Define a logging convention

We have not yet formalized a policy around logging, and I think this is clearly missing. It looks like a kind of convention is proposed in https://github.com/openzim/python-scraperlib/blob/main/src/zimscraperlib/logging.py but I do not find it very versatile and forces quite a lot of stuff (verbose dependencies, logger level is DEBUG, ...). Plus it seems to be rarely used in fact (I rarely encountered this at least).

I've read https://docs.python.org/3/howto/logging.html and there are tons of good ideas and recommendations, and this should probably be the first decision: adhere to recommendations in this document.

There are still open topics in this documentation, for which I propose decisions or suggest discussions below. WDYT? Anything else?

Live logging configuration

In order to be as versatile as possible, the software we produce must not take much decision on what needs to be logged and what is useless.

Software must hence allow to configure logging either:

• by an environment variable named LOGGING_LEVEL whose value is a log level which is set in the root handler
• by an environment variable named LOGGING_FILE_CONFIG whose value is the path to a configuration in configuration file format.

It should also support configuration by an environment variable named LOGGING_YAML_CONFIG whose value is the path to a YAML configuration adhering to the configuration dictionary schema

While LOGGING_YAML_CONFIG is the preferred solution (most versatile and less verbose), this is optional because it forces the use of PyYAML to parse the configuration file (and we might want to not include this in all projects).

If multiple LOGGING_* environment variables are set, the precedence is LOGGING_YAML_CONFIG > LOGGING_FILE_CONFIG > LOGGING_LEVEL.

We do not recommend to continue to support older environment variables for backward compatibility, or at least they must have even less precedence than the LOGGING_* ones.

Software must provide a sensible default configuration, typically with a default dict_config embedded in the code.

Loggers

Each software and library must use a unique logger with a unique and easily identifiable name, such as the __name__ of their top-level package or module. They must not log directly to the root logger.

This is meant to help the software / library user configure the logging verbosity or handlers as they wish while maintaining simplicity. Using a unique logger per software and library is deemed sufficient and avoids by default logging configurations that rely too heavily on module names. Should a specific need arise (e.g. filter out one kind of very verbose logs), it is always possible to achieve this with custom filters based on pathname or log content for instant. Experience shows we are usually lazy to do this anyway and long logger names are in fact just clogging the logs.