pytorch/examples

Resume will override learning rate on command line.

Open

#816 opened on Aug 19, 2020

View on GitHub
 (1 comment) (0 reactions) (0 assignees)Python (21,634 stars) (9,429 forks)batch import
good first issue

Description

If you resume from a checkpoint and want to run with a new learning rate, the saved learning rate will overwrite what you specify on the command line. The fix is simple.

Contributor guide

Tech stack
pythonpytorch
Domain
machine learning
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
under 1 hour
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Python programmingPyTorch basics
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
70
Research direction
Look at the training script where checkpoint loading is implemented. The issue is that the saved learning rate from the checkpoint overwrites the command line argument. Identify the line where the learning rate is restored and modify it to only set the learning rate from the checkpoint if no command line argument is provided. Ensure that the command line value takes precedence.