Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Is it not possible to set the rollout engine to multiple gpus?

Open
  • Is something wrong?
#470 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Stale
Tech stack
python
Domain
distributed-systems, machine-learning

Research direction

Start by reproducing the crash with tensor_parallel_size greater than 1 in the rollout engine while training remains configured for one GPU. Trace how the engine arguments reach the vllm rollout setup and identify the limitation. Done means multi-GPU rollouts work without requiring multi-GPU Unsloth training, with coverage for the relevant configuration.

Written by the indexing model from the issue text.

Description

question

When I set the 'tensor_parallel_size'>1 in the engine args the code crashes. I understand that the training itself with Unsloth only supports one gpu for now (though they have announced this is changing), but is it not possible to still perform the rollouts with the vllm engine across multiple gpus?

Dominant language
Python
Stars
10.8k
Forks
997
Avg merge
10h 1m
Merged PRs (30d)
117

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from OpenPipe/ART

All issues in OpenPipe/ART

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.