Lightning-AI/pytorch-lightning

Distributed mode overwrites the user's choice for dataloaders shuffling

开放

#10,170 创建于 2021年10月27日

 (8 条评论) (0 个反应) (0 位负责人)Python (3,233 个派生)batch import
data handlingfeaturehelp wanted

仓库指标

星标
 (26,687 个星标)
PR 合并指标
 (平均合并 9天 15小时) (30 天内合并 3 个 PR)

描述

🐛 Bug

I ordered my training data in a specific manner and passed it to the DataLoader with shuffle=False (I use reload_dataloaders_every_n_epochs=1 to control it every epoch). Then, I found out that running in distributed mode re-creates the train dataloader and always set shuffle=True. The opposite happens for the val dataloader.

I think that the re-creation of the dataloaders must be using the same shuffle setting the original dataloaders had. Is there any reason not to do so?


Here is the related code with the hard-coded shuffle values:

train - shuffle forced to be true https://github.com/PyTorchLightning/pytorch-lightning/blob/c33df2639f19d49c5e7520294e3221efe402d684/pytorch_lightning/trainer/data_loading.py#L330-L333

val - shuffle forced to be true https://github.com/PyTorchLightning/pytorch-lightning/blob/c33df2639f19d49c5e7520294e3221efe402d684/pytorch_lightning/trainer/data_loading.py#L442-L443

cc @borda @justusschock @awaelchli

贡献者指南