Easier change optimizer/learning rate instead of reading state_dict from checkpoint
#4044 aperta il 10 ott 2020
Metriche repository
- Star
- (26.687 star)
- Metriche merge PR
- (Merge medio 9g 15h) (3 PR mergiate in 30 g)
Descrizione
🚀 Feature
Easier change optimizer / learning rate without recoding
Motivation
Usually when we want to change the optimizer from one to another(for example,using Adam to fast init training and turn to SGD at the end) in PyTorch, we may have to load state_dict of the model and change then change the optimizer in the code.
In PytorchLightning, model state_dict, state of all optimizers, state of all learning rate schedulers etc. are automately saved in the lightning checkpoint. It seems that we will still have to load state_dict of the model from the checkpoint. (Similar issues also happen when we want to change the learning rate).
Is it possible that we are able to directly change the code and use a new optimizer/ new learning rate instead of recoding and reading the state_dict from checkpoint?
Pitch
Directly change the code and then a new optimizer/ new learning rate could be used.