verl-project/verl

[Bug] workflow `e2e_prime` is sometimes stuck

Open

#1344 opened on Apr 30, 2025

View on GitHub
 (1 comment) (0 reactions) (0 assignees)Python (21,533 stars) (3,940 forks)auto 404
bugcall for contributiongood first issue

Description

Motivation

The workflow e2e_prime is sometimes stuck and reaches timeout, while sometimes finishes normally. See https://github.com/volcengine/verl/actions/workflows/e2e_prime.yml for details.

Plan

  • Reproduce the issue locally with the container and commands used in the workflow.
  • Check where the root cause is.

Contributor guide

Tech stack
pythonshellgithub actions
Domain
ci cdtesting
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
fresh
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Familiarity with GitHub ActionsDocker experiencePython basicsDebugging CI pipelines
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
40
Research direction
First, examine the workflow file at .github/workflows/e2e prime.yml to understand the commands and environment. Then, attempt to replicate the issue locally using the same container image as in the workflow. Analyze the logs of both failed and successful runs (available at https://github.com/volcengine/verl/actions/workflows/e2e prime.yml) to identify where the process hangs. Look for common causes such as resource exhaustion, network timeouts, or race conditions in test setup.