pytorch/vision

LSUN caching is ugly

Open

#825 opened on Mar 29, 2019

View on GitHub
 (2 comments) (0 reactions) (0 assignees)Python (6,858 forks)batch import
enhancementhelp wanted

Repository metrics

Stars
 (15,050 stars)
PR merge metrics
 (Avg merge 12d 8h) (14 merged PRs in 30d)

Description

The cache for the LSUN dataset (136MB just for bedrooms) gets dumped into the same directory as the script you're running, with a filename that's a function of the path to the dataset itself. Generating the cache can take several hours, eg on a standard EC2 server with EBS disk. If you want to run a script in a different folder, you have to regenerate the cache (hours) or copy the cache file to the new folder. If you move the dataset to a different folder, you either regenerate the cache or try to modify the cache filename to match the new path. No matter what you have cache files side by side with your code that need special gitignore directives (they don't have an extension to use wildcards on).

It should be possible to put the cache file in the data directory or something to fix these issues.

Contributor guide

Research direction
Examine the LSUN dataset caching logic in torchvision/datasets/lsun.py and propose a change to store cache files in the data directory instead of the script directory.
Tech stack
pythonpytorch
Domain
computer vision
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Mostly clear
Prerequisites
PythonPyTorch
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
60