Hacktoberfest 2026：维护者为十月标记出来的 issue，仍然开放、适合新手。 浏览 Hacktoberfest issue

LocalBackend/Unsloth: gradient_accumulation_steps > 1 hangs the trainer (producer waits for one result per sequence)

未关闭
  • 哪里不对？
#1,121 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

@monsterlag 已经在做这个了。

开始于 2026年10月21日。

  • #1137 来自 @monsterlag —— 未关闭

评估

难度
4/5
预计耗时
3-5 天
新手友好度
15/100
Issue 类型
缺陷
描述清晰度
基本清楚
活跃度
活跃
技术栈
python
领域
machine-learning

调研方向

The hang is in the producer loop of run_unsloth_rl_training in art/unsloth/train.py, which pairs each inputs_queue put with a results_queue get, and in _async_prepare_inputs. Read how gradient_accumulation_steps reaches the trainer through TrainerArgs, then reproduce with gradient_accumulation_steps=2 to confirm the deadlock. Done means either the step runs to completion or a clear error is raised up front, and the loss weighting question in loss.py has a maintainer decision.

由索引模型根据 Issue 内容生成。

描述

Versions: openpipe-art 0.5.20 (LocalBackend, Unsloth service), unsloth 2026.3.3, ART's managed vLLM runtime 0.25.1, one GPU.

What happens: with trainer_args=art.dev.TrainerArgs(per_device_train_batch_size=1, gradient_accumulation_steps=2) the first backend.train(...) call never returns. The GPU is idle; py-spy shows the trainer thread waiting for its next input in _async_prepare_inputs (inputs_queue.get()), while run_unsloth_rl_training waits for a result.

Why (as far as I can tell from art/unsloth/train.py): run_unsloth_rl_training puts one packed sequence into inputs_queue and then awaits one item from results_queue before it puts the next sequence. A result is only produced after an optimizer step. With gradient_accumulation_steps=k > 1 the Hugging Face trainer needs k inputs before it steps, so after the first input both sides wait for each other.

So gradient_accumulation_steps is accepted through TrainerArgs but cannot work with this producer loop. If it is not meant to be supported, a clear error would help.

Why it matters for long rollouts: with agent rollouts of 10-25K tokens, every trajectory fills its own packed row, and with per_device_train_batch_size=1 (memory) every trajectory becomes its own optimizer step: 32 Adam updates per train() call for 4 groups x 8 rollouts, each seeing a single trajectory with a positive or negative advantage, instead of one update over the group.

A possible fix: in run_unsloth_rl_training, put gradient_accumulation_steps sequences into the queue before awaiting one result, and pad the last batch of a step with zero-weight rows (or raise if the row count is not a multiple).

Related observation: the loss multiplies each token by weights = 1 / (trained tokens of its trajectory) and then divides by assistant_mask.sum() of the row (loss.py, policy_loss * weights * assistant_mask, then / denominator). With several trajectories per row this gives every trajectory equal weight, but with one trajectory per row its loss is effectively divided by its length twice, so long trajectories get smaller gradients than short ones. Not necessarily a bug, but it surprised me.

Happy to send a PR for the producer loop if that is the direction you prefer.

主要语言
Python
星标
10.8k
派生
1k
平均合并
12 小时 33 分钟
30 天内合并 PR
126

环境准备

  • 没有 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

OpenPipe/ART 的其他 Issue

查看 OpenPipe/ART 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。