unslothai/unsloth

[Bug] Multi-gpu finetuning in Unsloth

Open

#3,248 opened on 2025年9月1日

GitHub で見る
 (2 comments) (0 reactions) (0 assignees)Python (5,658 forks)batch import
help wantedmultigpu

Repository metrics

Stars
 (64,271 stars)
PR merge metrics
 (平均マージ 3d 15h) (30d で 525 merged PRs)

説明

  1. Did you update? pip install --upgrade unsloth unsloth_zoo -->yes
  2. Colab or Kaggle or local / cloud -->cloud
  3. Number GPUs used, use nvidia-smi --?> 4
  4. Which notebook? Please link!
  5. Which Unsloth version, TRL version, transformers version, PyTorch version?
  6. Which trainer? SFTTrainer, GRPOTrainer etc --> SFTTrainer
Put Minimal code to reproduce error here ###Remove Hugging Face token###

🦥 You can also ask via our Reddit page: https://www.reddit.com/r/unsloth/

I am able to finetuned Gemma 3 4B model using unsloth in one gpu it is working fine, but when I am trying to use it in multi-gpu settings, I am not able to use it properly. Any help regarding multi-gpu support will be appriciate. Please help to set up multi-gpu finetuning through unsloth. Thanks

コントリビューターガイド