unslothai/unsloth

[Bug] Multi-gpu finetuning in Unsloth

Open

#3,248 创建于 2025年9月1日

在 GitHub 查看
 (2 评论) (0 反应) (0 负责人)Python (5,658 fork)batch import
help wantedmultigpu

仓库指标

Star
 (64,271 star)
PR 合并指标
 (平均合并 3天 15小时) (30 天内合并 525 个 PR)

描述

  1. Did you update? pip install --upgrade unsloth unsloth_zoo -->yes
  2. Colab or Kaggle or local / cloud -->cloud
  3. Number GPUs used, use nvidia-smi --?> 4
  4. Which notebook? Please link!
  5. Which Unsloth version, TRL version, transformers version, PyTorch version?
  6. Which trainer? SFTTrainer, GRPOTrainer etc --> SFTTrainer
Put Minimal code to reproduce error here ###Remove Hugging Face token###

🦥 You can also ask via our Reddit page: https://www.reddit.com/r/unsloth/

I am able to finetuned Gemma 3 4B model using unsloth in one gpu it is working fine, but when I am trying to use it in multi-gpu settings, I am not able to use it properly. Any help regarding multi-gpu support will be appriciate. Please help to set up multi-gpu finetuning through unsloth. Thanks

贡献者指南