Change in logging frequency for automated logging
#16,821 创建于 2023年2月20日
仓库指标
- Star
- (26,687 star)
- PR 合并指标
- (平均合并 9天 15小时) (30 天内合并 3 个 PR)
描述
Bug description
Utilising Automated Logging with self.log and self.log_dict as described in the documentation results in a shift of the logging frequency after various amounts of steps.

This is also observed in all train metrics but only in the _step. It could be mutually exclusive to the WandbLogger but has already been observed to some extent in the TensorboardLogger as reported in #13525 and more generally in #10436.
How to reproduce the bug
Call self.log(, on_step=True, on_epoch=True) in training_step and let it run for more than 15k steps (in my case).
The logging rate initial is equal to log_every_n_steps=50 for some iterations but jumps wildly around for others.
batch_size=10 (specified in self.log(batch_size=10))
test_subjects=20
samples_per_subject=10
This equals to 200 samples per epoch and 20 steps per epoch. Even if log_every_n_steps=50, this should log then precisely every 100 steps (as the first 50 is not met according to my understanding) and not jump around from 2 to 200.
Environment
- Lightning Component: Trainer/ LightningModule
- Python 3.9
- Pytorch-lightning 1.9.1 (installed with pip)
- PyTorch 1.13.1
- CUDA/ cuDNN: cuda11.6-cudnn8
- OS: Linux (Kernel 3.10.0-1160)
- Running environment: server