pytorch/examples

mnist example contains bad practices leading to wrongful results

Open

#623 opened on Aug 27, 2019

View on GitHub
 (2 comments) (0 reactions) (0 assignees)Python (9,429 forks)batch import
good first issue

Repository metrics

Stars
 (21,634 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

In the mnist example there should be explicit comments about the default nature of the F.nll_loss which is to average the results over the mini-batch. The other thing that can lead to errors is the len(train_loader.dataset) even though that returns the number of samples in the train dataset if someone was to create a split of train data into train/valid using SubsetRandomSampler then calling again len(train_loader.dataset) will still return you the full number of samples in the train set (which is wrong) without reflecting the correct number of samples since the train set now contains X% less samples which have been kept as validation set.

Contributor guide

Research direction
Update the MNIST example comments to clarify that F.nll loss averages over the mini batch by default and that len(train loader.dataset) gives the full dataset size even when using SubsetRandomSampler.
Tech stack
python
Domain
machine learningai
Issue type
Documentation
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
1
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Under 1 hour
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
PyTorch
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
90