Add a method to initialize dynamic modules from state_dict
#1 119 ouverte le 6 sept. 2022
Métriques du dépôt
- Stars
- (1 516 stars)
- Métriques de merge PR
- (Aucune PR mergée en 30 j)
Description
Hi,
Training
I initialize net=MTSimpleCNN() and train this multi-task model with multi-head classifier on two tasks (one task per experience). After training the model has two heads.
I save the model this model's state dict using standard pytorch torch.save({'state_dict': net.state_dict()}, filename). and I want to evaluate it later using a different script.
Evaluation
I newly initialize the model net=MTSimpleCNN() and load the weights using standard pytorch net.load_state_dict(torch.load(filename,map_location=device)['state_dict']). However the saved model state dict has two heads and the newly initialized one only one head. Keys don't match so I get this error:
Error(s) in loading state_dict for MTSimpleCNN: Unexpected key(s) in state_dict: "classifier.active_units_T0", "classifier.active_units_T1", "classifier.classifiers.1.active_units", "classifier.classifiers.1.classifier.weight", "classifier.classifiers.1.classifier.bias", "classifier.classifiers.0.active_units"
is there any easy way to initialize this multi-head network with arbitrary number of heads?
Cheers, Woj