create_lr_scheduler_with_warmup does not change init_lr to proper value

Hi,

In order to get expected sequence of lrs from create_lr_scheduler_with_warmup's scheduler, one must not attach it to engine on event EPOCH_COMPLETED because it produces the lr passed to optimizer's constructor at the beginning and then warmup lrs. This hurts warmup procedure. As a workaround, one could use event EPOCH_STARTED but it might not be a good solution.

It seems there should be something like below in line 1017 of param_scheduler.py within the for loop .

param_group['lr'] = warmup_start_value

To reproduce current behaviour:

param = nn.Parameter(torch.Tensor([3.]))
optimizer = torch.optim.SGD([param], lr=1e-3)
scheduler = StepLR(optimizer, 3)
with_warmup_scheduler = create_lr_scheduler_with_warmup(scheduler, warmup_start_value=1e-5, warmup_duration=3)

def process_func(e,b):
  param.grad = torch.Tensor([1.])
  optimizer.step()
trainer = Engine(process_func)
@trainer.on(Events.EPOCH_COMPLETED)
def _():
  print(op.param_groups[0]['lr'])
trainer.add_event_handler(Events.EPOCH_COMPLETED, with_warmup_scheduler)

output:

0.001
1e-05
0.000505
0.001
0.001
0.001
0.0001
0.0001
0.0001
1e-05