Lightning-AI/pytorch-lightning

Support saving and loading remote paths with FSDP

Open

#18,786 建立於 2023年10月12日

在 GitHub 查看
 (5 留言) (1 反應) (0 負責人)Python (3,233 fork)batch import
checkpointingfeaturehelp wantedstrategy: fsdp

倉庫指標

Star
 (26,687 star)
PR 合併指標
 (平均合併 9天 15小時) (30 天內合併 3 個 PR)

描述

Bug description

FSDPStrategy.load_checkpoint casts checkpoint_path to a pathlib.Path here. This will bork URIs, such as cloud checkpoint paths, e.g. s3://....

Example:

from pathlib import Path

checkpoint_path = "s3://asd/asd"
assert Path(checkpoint_path).as_posix() == checkpoint_path

NOTE: I am reporting this merely by looking at the source code; I have yet to confirm this with a test.

What version are you seeing the problem on?

master

How to reproduce the bug

from lightning.pytorch.strategies import FSDPStrategy

FSDPStrategy(...).load_checkpoint("s3://my/checkpoint")

Error messages and logs

I believe this exception will be raised.

Environment

#- Lightning Component (e.g. Trainer, LightningModule, LightningApp, LightningWork, LightningFlow):
#- PyTorch Lightning Version (e.g., 1.5.0):
#- Lightning App Version (e.g., 0.5.2):
#- PyTorch Version (e.g., 2.0):
#- Python version (e.g., 3.9):
#- OS (e.g., Linux):
#- CUDA/cuDNN version:
#- GPU models and configuration:
#- How you installed Lightning(`conda`, `pip`, source):
#- Running environment of LightningApp (e.g. local, cloud):

More info

No response

cc @borda @awaelchli @carmocca

貢獻者指南