Finetune-from-model rests learning rate, is there a way to finetune without resetting it?
Contributor guide
Tech stack
pythonpytorch
Domain
backendmachine learning
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
fresh
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
mostly clear
Prerequisites
fairseq basicslearning rate scheduling
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
60
Research direction
Investigate the finetune from model command in fairseq to understand how the learning rate is reset. Consider adding a command line flag (e.g., preserve lr) to retain the original learning rate from the checkpoint. Look at the training loop in fairseq/trainer.py and relevant configurations in fairseq/optim/ to implement the option.
