Lightning-AI/pytorch-lightning

Allow obtaining num_nodes from ClusterEnvironment

Open

#7,361 建立於 2021年5月4日

在 GitHub 查看
 (7 留言) (0 反應) (0 負責人)Python (3,233 fork)batch import
environmentfeaturegood first issuehelp wanted

倉庫指標

Star
 (26,687 star)
PR 合併指標
 (平均合併 9天 15小時) (30 天內合併 3 個 PR)

描述

🚀 Feature

num_nodes must currently be specified manually by the user. However, the number of nodes is generally known in a cluster environment [1] and could be provided by and initialized from ClusterEnvironment

[1] For example $AWS_BATCH_JOB_NUM_NODES in https://docs.aws.amazon.com/batch/latest/userguide/multi-node-parallel-jobs.html

Pitch

https://github.com/PyTorchLightning/pytorch-lightning/blob/763a9a9495977b23cbd6a57f10253b662fd592a5/pytorch_lightning/plugins/training_type/ddp.py#L62-L75

could be updated to initialize num_nodes from ClusterEnvironment if ClusterEnvironment is provided and implements a num_nodes method.

cc @borda @awaelchli @ananthsub

貢獻者指南