pytorch/ignite

Support for TorchSnapshot for efficient checkpoint saving and loading

Open

Aperta il 24 ott 2022

Vedi su GitHub
 (3 commenti) (0 reazioni) (0 assegnatari)Python (4313 star) (602 fork)batch import
enhancementhelp wanted

Descrizione

🚀 Feature

TorchSnapshot is a performant, memory-efficient checkpointing library for PyTorch applications, designed with large, complex distributed workloads in mind. It includes many optimizations to control for memory usage and optimize checkpoint writing for DDP-style workloads over torch.save/torch.load. For more information, please check out the readme: https://github.com/pytorch/torchsnapshot#why-torchsnapshot

This could be a nice addition to Ignite, similar to the existing Checkpoint handler

cc @yifuwang

Guida contributor

Tech stack
python
Dominio
machine learning
Tipo issue
feature
DifficoltàQuanto dovrebbe essere impegnativa per un nuovo contributor.
3
Tempo stimatoTempo stimato per completare e verificare uno scope piccolo.
3-5 days
Stato attivitàQuanto è probabile che la issue sia ancora attiva e reviewable.
stale
ChiarezzaQuanto chiaramente la issue descrive problema, scope e risultato atteso.
clear
Prerequisiti
PyTorch Ignite basicsUnderstanding of checkpointing concepts
Adatta ai principiantiQuanto la issue è adatta a contributor alla prima esperienza.
30
Direzione di ricerca
Start by reviewing the existing Checkpoint handler in pytorch/ignite (likely in ignite/handlers/checkpoint.py) to understand the current implementation. Then explore the TorchSnapshot documentation and examples to understand its API and how it could replace or augment the current checkpointing. Consider contributing by implementing a new handler or modifying the existing one to support TorchSnapshot, similar to how Ignite handles other saving methods. Discuss design in the issue comments with maintainers before proceeding.