Training can admit batches beyond trainer.total_epochs

Abierto
  • ¿Algo no está bien?
#693 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 3 días

@liuzhengyang699 ya está trabajando en esto.

Desde el 6/11/2026.

  • #694 de @liuzhengyang699 — abierto

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
30/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Estancado
Stack tecnológico
python
Área
machine-learning

Línea de trabajo

Start with AgentLightningRayPPOTrainer.fit() and _next_train_batch_dict() in agentlightning/verl/trainer.py, then trace the asynchronous sized refill helper described in the issue. The body says CPU regressions are ready; run the focused tests for sync and async epoch limits, partial refills, carry-over groups, step limits, and final validation/checkpointing. Done means no new inputs are admitted past the epoch limit, admitted async groups drain, and the explicit step limit takes precedence.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

With trainer.total_epochs=1 and a larger trainer.total_training_steps, synchronous training consumes the first batch of a second epoch before stopping. The outer fit() check runs before _next_train_batch_dict() discovers exhaustion; that helper increments the epoch and immediately opens the dataloader again.

Minimal CPU reproduction on bc3a9e8:

from types import SimpleNamespace
from agentlightning.verl.trainer import AgentLightningRayPPOTrainer

trainer = object.__new__(AgentLightningRayPPOTrainer)
trainer.config = SimpleNamespace(trainer=SimpleNamespace(total_epochs=1))
trainer.epoch = 1
trainer.train_dataloader = [{"item": [0, 1]}, {"item": [2, 3]}]
trainer._train_dataloader_iter = None
print([trainer._next_train_batch_dict()["item"] for _ in range(3)])
# [[0, 1], [2, 3], [0, 1]]

Only the first two batches belong to the requested epoch. A regression through fit() confirms the extra optimizer update when the step limit does not stop it first.

Asynchronous collection has the same unbounded iterator in its sized refill helper. A refill can cross the final epoch, while stopping immediately at the outer epoch check can leave previously admitted groups unfinished. The intended behavior is to stop admitting new inputs at the epoch limit and drain those groups, with the explicit step limit still taking precedence.

A focused fix and CPU regressions are ready. Nine of the initial 11 cases fail on unchanged main. Tests cover synchronous and asynchronous training, partial final refills, carry-over groups, independent step limits, and final validation/checkpointing. Checkpoint-resume epoch accounting is outside this change.

This was found during a follow-up audit with AI assistance. The reproduction and tests do not run GPU workers or end-to-end training.

Lenguaje dominante
Python
Estrellas
18.9k
Forks
1.7k
Merge medio
5 d 19 h
PR fusionados (30 d)
14

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de microsoft/agent-lightning

Todos los issues de microsoft/agent-lightning

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.