Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Distributed gather crashes on PyTorch < 2.6

Open Beginner friendly
  • Is something wrong?
#13,074 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@akshatvishu is already working on this.

Since Feb 3, 2026.

  • #13075 by @akshatvishu — open
  • #13093 by @Mr-Neutr0n — closed without merging
  • #13514 by @Ricardo-M-L — closed without merging
  • #13993 by @Whning0513 — closed without merging
  • #14515 by @Ricardo-M-L — closed without merging

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
68/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Stale
Tech stack
python, pytorch
Domain
distributed-systems

Research direction

Start in src/diffusers/models/_modeling_parallel.py at gather_size_by_comm and run the minimal distributed reproduction from the issue with PyTorch 2.4. Trace how the communication backends determine gather_device. Done means distributed gather no longer raises AttributeError on supported PyTorch 2.1–2.5 while retaining behavior on newer versions.

Written by the indexing model from the issue text.

Description

bug
Describe the bug

AttributeError: module 'torch' has no attribute 'accelerator' when running distributed gather on PyTorch versions < 2.6.

This error happens because gather_size_by_comm in src/diffusers/models/_modeling_parallel.py uses torch.accelerator.current_accelerator(), which only exists in PyTorch 2.6+. Diffusers officially supports PyTorch 2.1+, so this causes a crash on versions 2.1–2.5 with AttributeError: module 'torch' has no attribute 'accelerator'.

Reproduction

Since this is a utility function; it can be triggered directly with a minimal distributed setup:

import torch.distributed as dist
from diffusers.models._modeling_parallel import gather_size_by_comm

dist.init_process_group(
    backend="gloo",
    init_method="file:///tmp/pg",
    rank=0,
    world_size=1,
)

gather_size_by_comm(1, dist.group.WORLD)
Logs
[rank0]: Traceback (most recent call last):
[rank0]:   File "/home/aja/diffusers/test.py", line 11, in <module>
[rank0]:     gather_size_by_comm(1, dist.group.WORLD)
[rank0]:   File "/home/aja/diffusers/src/diffusers/models/_modeling_parallel.py", line 293, in gather_size_by_comm
[rank0]:     gather_device = "cpu" if "cpu" in comm_backends else torch.accelerator.current_accelerator()
[rank0]:                                                          ^^^^^^^^^^^^^^^^^
[rank0]:   File "/home/aja/diffusers/.venv/lib/python3.11/site-packages/torch/__init__.py", line 2216, in __getattr__
[rank0]:     raise AttributeError(f"module '{__name__}' has no attribute '{name}'")
[rank0]: AttributeError: module 'torch' has no attribute 'accelerator'
System Info

diffuser : 0.37.0.dev0
torch : 2.4.0
python : 3.11
system: Linux

Who can help?

@sayakpaul @DN6

Dominant language
Python
Stars
34.6k
Forks
7.4k
Avg merge
4d 16h
Merged PRs (30d)
44

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from huggingface/diffusers

All issues in huggingface/diffusers

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.