tracel-ai/burn
CosineAnnealingLrScheduler displays non-standard behavior
Geschlossen
#5.128 geöffnet am 06.07.2026
buggood first issue
Repository-Metriken
- Stars
- (15.550 Sterne)
- PR-Merge-Metriken
- (Durchschn. Merge 2T 22h) (129 gemergte PRs in 30 T)
Beschreibung
I'm using the following setup:
let cosine_config = CosineAnnealingLrSchedulerConfig::new(max_lr, 250)
.with_min_lr(min_lr);
where max_lr = 1e-3 and min_lr = 1e-6
And get the following recurring half-cosine LR waveform:
For pytorch, documentation suggests that once the cosine wave reaches its minima, the LR remains there for the duration (i.e. without restarting the cycle - although I'm unsure what happens in the case where the minima approaches 0.0 which would be an unfortunate place to continue training, as it would be for Burn without configuring a .with_min_lr(), as the default minima would be 0.0 also).
Burn's current implementation which cycles a half cosine wave between max and min does not seem like an acceptable or expected outcome, and clearly could contribute to training instability.