verl-project/verl

[FR] Better metric reduction utility

Open

#1508 opened on May 13, 2025

View on GitHub
 (2 comments) (2 reactions) (0 assignees)Python (21,533 stars) (3,940 forks)auto 404
enhancementfeature requestgood first issue

Description

Motivation

reduce_metrics seems to be used for reducing metrics across DP ranks and batches, which only simply means the values.

However, reducing metrics across DP ranks is non-trivial since some metrics like "mean" are not linear.

Contributor guide

Tech stack
pythonpytorch
Domain
machine learningdata
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
4
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
fresh
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
mostly clear
Prerequisites
PythonPyTorch distributedverl codebase
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
25
Research direction
Investigate the current reduce metrics function at https://github.com/verl project/verl/blob/main/verl/utils/metric/utils.py. Understand the different types of metrics and their reduction requirements across data parallel ranks. Propose a generalized reduction strategy that accounts for non linear metrics like mean. Consider existing implementations in PyTorch or other frameworks.