Lightning-AI/pytorch-lightning

Redesign dataloader re-instantiation context managers for Lite

Open

#15,179 建立於 2022年10月18日

在 GitHub 查看
 (10 留言) (0 反應) (1 負責人)Python (3,233 fork)batch import
discussionfabrichelp wantedpriority: 2refactor

倉庫指標

Star
 (26,687 star)
PR 合併指標
 (平均合併 9天 15小時) (30 天內合併 3 個 PR)

描述

Proposed refactor

Redesign the context managers that enable re-instantiations of DataLoader after #14992 deprecates the run method design.

Motivation

We currently have two context managers over the run method:

https://github.com/Lightning-AI/lightning/blob/776432fd7e34cc02e9dbfb16ad8ec8491c176dbf/src/lightning_lite/lite.py#L394-L397

  • _replace_dunder_methods(DataLoader, "dataset")
  • _replace_dunder_methods(BatchSampler)

These get applied for the user if they implement the run method. However, this won't be the case anymore after #14992. We need to find an alternative approach.

Pitch

We probably need to call these methods at the start of the program, for example in LightningLite.__init__. But it is unclear how to undo the patching which is usually done in __exit__. Happy to hear suggestions.

cc @borda @justusschock @awaelchli @tchaton @carmocca @otaj

貢獻者指南