Lightning-AI/pytorch-lightning

Redesign dataloader re-instantiation context managers for Lite

Open

#15,179 创建于 2022年10月18日

在 GitHub 查看
 (10 评论) (0 反应) (1 负责人)Python (3,233 fork)batch import
discussionfabrichelp wantedpriority: 2refactor

仓库指标

Star
 (26,687 star)
PR 合并指标
 (平均合并 9天 15小时) (30 天内合并 3 个 PR)

描述

Proposed refactor

Redesign the context managers that enable re-instantiations of DataLoader after #14992 deprecates the run method design.

Motivation

We currently have two context managers over the run method:

https://github.com/Lightning-AI/lightning/blob/776432fd7e34cc02e9dbfb16ad8ec8491c176dbf/src/lightning_lite/lite.py#L394-L397

  • _replace_dunder_methods(DataLoader, "dataset")
  • _replace_dunder_methods(BatchSampler)

These get applied for the user if they implement the run method. However, this won't be the case anymore after #14992. We need to find an alternative approach.

Pitch

We probably need to call these methods at the start of the program, for example in LightningLite.__init__. But it is unclear how to undo the patching which is usually done in __exit__. Happy to hear suggestions.

cc @borda @justusschock @awaelchli @tchaton @carmocca @otaj

贡献者指南