Lightning-AI/pytorch-lightning

Allow obtaining num_nodes from ClusterEnvironment

開放

#7,361 建立於 2021年5月4日

 (8 則留言) (0 個反應) (0 位負責人)Python (3,233 個分叉)batch import
environmentfeaturegood first issuehelp wanted

倉庫指標

星標
 (26,687 顆星)
PR 合併指標
 (平均合併 9天 15小時) (30 天內合併 3 個 PR)

描述

🚀 Feature

num_nodes must currently be specified manually by the user. However, the number of nodes is generally known in a cluster environment [1] and could be provided by and initialized from ClusterEnvironment

[1] For example $AWS_BATCH_JOB_NUM_NODES in https://docs.aws.amazon.com/batch/latest/userguide/multi-node-parallel-jobs.html

Pitch

https://github.com/PyTorchLightning/pytorch-lightning/blob/763a9a9495977b23cbd6a57f10253b662fd592a5/pytorch_lightning/plugins/training_type/ddp.py#L62-L75

could be updated to initialize num_nodes from ClusterEnvironment if ClusterEnvironment is provided and implements a num_nodes method.

cc @borda @awaelchli @ananthsub

貢獻者指南