[Bug] run.py: LOCAL_RANK defaults to 1 instead of 0

Open Beginner friendly
#1,589 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
1/5
Estimated time
Under an hour
Newbie friendliness
78/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Quiet
Tech stack
python
Domain
distributed-systems

Research direction

Start in run.py around the LOCAL_RANK definition at line 36 and compare its fallback with the other distributed environment variables. Change the fallback to the identity value described in the issue, then verify that rank 0 computes the correct GPU slice when LOCAL_RANK is absent.

Written by the indexing model from the issue text.

Description

Summary

In run.py (confirmed on main @ 224943f), LOCAL_RANK defaults to 1:

RANK = int(os.environ.get('RANK', 0))
WORLD_SIZE = int(os.environ.get('WORLD_SIZE', 1))
LOCAL_WORLD_SIZE = int(os.environ.get("LOCAL_WORLD_SIZE", 1))
LOCAL_RANK = int(os.environ.get("LOCAL_RANK", 1))   # <-- should default to 0

https://github.com/open-compass/VLMEvalKit/blob/224943f9b5833f8dcb5b5040de64cfcdd59a3090/run.py#L36

Every other distributed variable defaults to its identity value (RANK=0, WORLD_SIZE=1, LOCAL_WORLD_SIZE=1). LOCAL_RANK defaulting to 1 is inconsistent and incorrect — a local rank starts at 0.

Impact

LOCAL_RANK is used to compute the per-process GPU slice:

DEVICE_START_IDX = GPU_PER_PROC * LOCAL_RANK

If a multi-process run sets LOCAL_WORLD_SIZE but does not export LOCAL_RANK for rank 0 (e.g. a custom launcher rather than torchrun), process 0 computes DEVICE_START_IDX = GPU_PER_PROC * 1 and binds to the wrong GPU slice; the highest-rank process can index past the GPU list. On a single-GPU machine there is no device index >= 1.

Fix

LOCAL_RANK = int(os.environ.get("LOCAL_RANK", 0))

Happy to open a one-line PR.

Dominant language
Python
Stars
4.4k
Forks
768
Avg merge
1d 20h
Merged PRs (30d)
20

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from open-compass/VLMEvalKit

All issues in open-compass/VLMEvalKit

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.