[Question] Details about DAPO training (entropy behavior, clip_high, dynamic sampling, loss design)
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Documentation
- Clarity
- Needs clarification
- Activity status
- Quiet
- Tech stack
- python
- Domain
- machine-learning
Research direction
Start by reading the DAPO description in the paper and comparing it with the released training scripts. Trace whether clip_high, dynamic sampling, loss formulation, length normalization, and entropy stabilization are represented. Done means providing a maintainer-confirmed explanation of the configuration used in the paper and how it relates to the observed entropy decrease.
Written by the indexing model from the issue text.
Description
Hi, thanks again for your great work!
I have a question regarding the DAPO training described in the paper.
In the paper, it is mentioned that using DAPO leads to a noticeable entropy decrease during training. However, from my understanding, one of the key motivations behind DAPO is to prevent entropy from collapsing too quickly and maintain better exploration.
While going through the released training scripts, I did not find implementations corresponding to some components that seem important for DAPO, such as:
clip_highin the policy update- dynamic filtering of samples
- specific mechanisms to stabilize entropy
So I would like to ask for more detailed clarification on the DAPO setup used in the paper:
-
Was
clip_highused in your implementation?
If so, what value/range was used? -
Was any form of dynamic sampling applied during training?
-
What loss formulation was used?
- Is it token-level mean loss (
token_mean) or sequence-level? - Any modification compared to standard PPO-style objectives?
- Is it token-level mean loss (
-
Was length normalization or length penalty applied?
If yes, how was it incorporated into the reward or loss? -
How do you interpret the entropy drop observed in your experiments?
Is it expected behavior under your configuration, or controlled in some way?
Understanding these details would be very helpful for reproducing the results and better understanding the role of DAPO in your framework.
Thanks a lot for your time and help!
- Dominant language
- Python
- Stars
- 166
- Forks
- 5
- PR merge metrics
- No merged PRs in 30d
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from EvolvingLMMs-Lab/OpenMMReasoner
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
EvolvingLMMs-Lab/OpenMMReasoner#12 · 1 comment ·
-
training timeOpen
Difficulty 1/5 Under an hour Newbie friendliness 20/100
EvolvingLMMs-Lab/OpenMMReasoner#9 · 1 comment ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
EvolvingLMMs-Lab/OpenMMReasoner#8 · 2 comments ·
-
Difficulty 1/5 Under an hour Newbie friendliness 20/100
EvolvingLMMs-Lab/OpenMMReasoner#7 · 1 comment ·
All issues in EvolvingLMMs-Lab/OpenMMReasoner
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
google-agentic-commerce/AP2#388 ·
-
area:codegen backend:cuda backend:hip bug direction:input
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
CrossGL/crosstl#3061 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
rlaope/oh-my-hermes#2304 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
mochan-tk/agentic-dev-kit-for-codex#64 ·
Maintainers usually reply within 1 day
-
bug docs performance
Difficulty 1/5 Under an hour Newbie friendliness 85/100
Maintainers usually reply within 3 days