Easier change optimizer/learning rate instead of reading state_dict from checkpoint

🚀 Feature

Easier change optimizer / learning rate without recoding

Motivation

Usually when we want to change the optimizer from one to another(for example,using Adam to fast init training and turn to SGD at the end) in PyTorch, we may have to load state_dict of the model and change then change the optimizer in the code.

In PytorchLightning, model state_dict, state of all optimizers, state of all learning rate schedulers etc. are automately saved in the lightning checkpoint. It seems that we will still have to load state_dict of the model from the checkpoint. (Similar issues also happen when we want to change the learning rate).

Is it possible that we are able to directly change the code and use a new optimizer/ new learning rate instead of recoding and reading the state_dict from checkpoint?

Pitch

Directly change the code and then a new optimizer/ new learning rate could be used.

Alternatives

Additional context