pytorch/ignite

`Timer`'s misleading behaviour when epoch completion time calculated

Open

#2,157 opened on Aug 11, 2021

View on GitHub
 (7 comments) (1 reaction) (1 assignee)Python (4,313 stars) (602 forks)batch import
HacktoberfestPyDataGlobalenhancementhelp wanted

Description

When I tried calculating the time taken to complete a single epoch via Timer, the handlers attached to trainer before Timer were executed first, and thus their time also got recorded by the Timer too for a single epoch. Therefore the true time taken for epoch completion, provided by trainer.state.times, is less than what Timer calculated. This can be misleading. More clarification on how this actually works in the docs would be appreciated. Or the Timer's functionality can be enhanced to step before all other handlers attached to an event could also be helpful. Notebook to quickly verify this here.

Contributor guide

Tech stack
pythonpytorch
Domain
machine learningdeveloper experience
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Familiarity with PyTorch Ignite event system
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
70
Research direction
Examine the Timer implementation in PyTorch Ignite, specifically how it hooks into events. The issue suggests that Timer should step before other handlers. Check the source code for the Timer class and its event registration. Consider whether a simple reordering of handler execution or a new event attribute is needed. The provided Colab notebook can be used to reproduce the issue.