Lightning-AI/pytorch-lightning
An example for knowledge distillation. Especially for the load the teacher model's weights , and train the student model
Offen
#17.592 geöffnet am 08.05.2023
examplefeaturehelp wanted
Repository-Metriken
- Stars
- (26.687 Sterne)
- PR-Merge-Metriken
- (Durchschn. Merge 9T 15h) (3 gemergte PRs in 30 T)
Beschreibung
Description & Motivation
No response
Pitch
A example for knowledge distillation. Especially for the load the teacher model's weight ,and train the student model.
Now I have a trained teacher_model. When I want to train the student model, I need to load the teacher model's weights and set the teacher model to eval().
I am confused about the specific steps to do above using the Lightning. Could I get an example for it?
Alternatives
An example for knowledge distillation. Especially for the load the teacher model's weight ,and train the student model
Additional context
An example for knowledge distillation. Especially for the load the teacher model's weight ,and train the student model.
cc @borda