Add support for OpenMPI Rank Assignment · facebookresearch/fairseq#4645

(0 comments) (0 reactions) (0 assignees)Python (6.224 forks)batch import

enhancementhelp wantedneeds triage

Métricas do repositório

Stars: (29.107 stars)
Métricas de merge de PR: (Nenhuma PRs mesclada em 30d)

Description

🚀 Feature Request

Currently, there is no way to read the variables OMPI_COMM_WORLD_SIZE, OMPI_COMM_WORLD_RANK, and OMPI_COMM_WORLD_LOCAL_RANK to assign ranks to any individual processes launched with OpenMPI

Motivation

I'd like to launch jobs with OpenMPI without using multiple-data mode, which can cause issues with command length if I have to specify all of the ranks manually.

Pitch

Add _infer_ompi_init function in fairseq/distributed/utils.py which reads the environment variables:

MASTER_ADDR
MASTER_PORT
OMPI_COMM_WORLD_SIZE
OMPI_COMM_WORLD_RANK
OMPI_COMM_WORLD_LOCAL_RANK

and updates the distributed config accordingly.

Alternatives

N/A

Additional context

N/A

Guia do colaborador

Direção de pesquisa: Examine a função infer slurm init existente em fairseq/distributed/utils.py para referência de padrão e implemente infer ompi init lendo as variáveis de ambiente OpenMPI especificadas e atualizando a configuração distribuída de acordo.
Pilha de tecnologia: python
Domain: backend
Tipo Issue: Funcionalidade
Difficulty: 2
Tempo estimado: 1-3 horas
Status da atividade: Ativo
Clarity: Claro
Prerequisites: PythonGit
Simpatia para novatos: 75