tracel-ai/burn
CosineAnnealingLrScheduler displays non-standard behavior
Chiusa
#5128 aperta il 6 lug 2026
buggood first issue
Metriche repository
- Star
- (15.550 stelle)
- Metriche merge PR
- (Merge medio 2g 22h) (129 PR mergiate in 30 g)
Descrizione
I'm using the following setup:
let cosine_config = CosineAnnealingLrSchedulerConfig::new(max_lr, 250)
.with_min_lr(min_lr);
where max_lr = 1e-3 and min_lr = 1e-6
And get the following recurring half-cosine LR waveform:
For pytorch, documentation suggests that once the cosine wave reaches its minima, the LR remains there for the duration (i.e. without restarting the cycle - although I'm unsure what happens in the case where the minima approaches 0.0 which would be an unfortunate place to continue training, as it would be for Burn without configuring a .with_min_lr(), as the default minima would be 0.0 also).
Burn's current implementation which cycles a half cosine wave between max and min does not seem like an acceptable or expected outcome, and clearly could contribute to training instability.