verl-project/verl

[FR] Tests for gradient accumulation

Open

#907 创建于 2025年4月3日

在 GitHub 查看
 (0 评论) (0 反应) (0 负责人)Python (21,533 star) (3,940 fork)auto 404
enhancementgood first issue

描述

Summary

Promise from #906.

Add tests to assert the losses on the same mini-batch are equal

  • between w/o & w/ gradient accumulation
  • using all the different loss aggregation modes

Plan

We might went to test end-to-end to avoid potential implication by any other changes.

Plan Pro
A) Inherit a new Trainer class to test Standalone without affecting the main config
B) Add a config term and modify update_* to test 1) Minimal code modification 2) Compatibility to any Trainer

贡献者指南

技术栈
pythonpytorch
领域
testing
议题类型
feature
难度面向新贡献者的预计实现难度，1 表示很小改动，5 表示专家级工作。
3
预计时间有经验贡献者完成调查、实现、测试并准备 pull request 的粗略时间范围。
half day
活动状态议题当前的可参与程度：新鲜、活跃、陈旧、阻塞或等待维护者输入。
fresh
清晰度议题是否清楚说明期望改动、验收标准和下一步。
clear
前置要求
PythonPyTorchgradient accumulation
新手友好度1-100 的估计分数，表示该议题对首次贡献者的友好程度。
70
研究方向
Investigate the two proposed plans: A) creating a new Trainer subclass for testing to avoid affecting main config, or B) adding a config term and modifying update * methods. The tests should assert that losses on the same mini batch are equal with and without gradient accumulation across all loss aggregation modes. Look at the existing test suite in the repository to understand the testing structure and consider implementing both options if feasible. Examine the gradient accumulation implementation in the trainer to identify the update * methods and loss aggregation modes.
[FR] Tests for gradient accumulation · verl-project/verl#907 | Good First Issue