enhancementhelp wantedneeds triage
Repository-Metriken
- Stars
- (29.107 Stars)
- PR-Merge-Metriken
- (Keine gemergten PRs in 30 T)
Beschreibung
🚀 Feature Request
Currently, there is no way to read the variables OMPI_COMM_WORLD_SIZE, OMPI_COMM_WORLD_RANK, and OMPI_COMM_WORLD_LOCAL_RANK to assign ranks to any individual processes launched with OpenMPI
Motivation
I'd like to launch jobs with OpenMPI without using multiple-data mode, which can cause issues with command length if I have to specify all of the ranks manually.
Pitch
Add _infer_ompi_init function in fairseq/distributed/utils.py which reads the environment variables:
MASTER_ADDR
MASTER_PORT
OMPI_COMM_WORLD_SIZE
OMPI_COMM_WORLD_RANK
OMPI_COMM_WORLD_LOCAL_RANK
and updates the distributed config accordingly.
Alternatives
N/A
Additional context
N/A