Easier change optimizer/learning rate instead of reading state_dict from checkpoint
#4 044 ouverte le 10 oct. 2020
Métriques du dépôt
- Stars
- (26 687 stars)
- Métriques de merge PR
- (Merge moyen 9j 15h) (3 PRs mergées en 30 j)
Description
🚀 Feature
Easier change optimizer / learning rate without recoding
Motivation
Usually when we want to change the optimizer from one to another(for example,using Adam to fast init training and turn to SGD at the end) in PyTorch, we may have to load state_dict of the model and change then change the optimizer in the code.
In PytorchLightning, model state_dict, state of all optimizers, state of all learning rate schedulers etc. are automately saved in the lightning checkpoint. It seems that we will still have to load state_dict of the model from the checkpoint. (Similar issues also happen when we want to change the learning rate).
Is it possible that we are able to directly change the code and use a new optimizer/ new learning rate instead of recoding and reading the state_dict from checkpoint?
Pitch
Directly change the code and then a new optimizer/ new learning rate could be used.