Carry terminal truncation metadata through Gym token-capture receipts
Maintainers usually reply within 2 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 30/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- python
- Domain
- machine-learning
Research direction
Start in nemo_rl/experience/rollout_reassembler.py, at receipt finalization near line 785, where truncation is currently inferred from the assembled length. Trace how the terminal response's incomplete_details.reason reaches that code, and how the TRUNCATED column feeds overlong_filtering. Done means the zero-token exhausted turn, sub-limit length exhaustion, completed, content-filtered and legacy receipt cases are covered by RolloutReassembler tests that check both the TRUNCATED value and the overlong filter result.
Written by the indexing model from the issue text.
Description
PR #4491 fixes multi-turn context exhaustion for ordinary Gym results, but the token-capture receipt path still marks truncation only when the assembled sequence length equals the maximum length. A terminal response that ends with incomplete_details.reason=max_output_tokens before that equality can therefore escape overlong_filtering and undercount truncation metrics.
Carry the terminal response's truncation evidence through receipt assembly and use the same context-exhaustion semantics when finalizing captured rows. Preserve compatibility with older receipts that lack the metadata, and keep content-filter or ordinary completed responses distinct from length truncation.
Add tests that drive receipt assembly through RolloutReassembler for a zero-token context-exhausted final turn, terminal length exhaustion below the total limit, an ordinary completed response, a content-filtered response, and a legacy receipt. Check both the emitted TRUNCATED column and the downstream overlong filter.
Current source: receipt finalization. This is the receipt-mode follow-up requested in the review of #4491; it is distinct from #2163's older missing-column error.
- Dominant language
- Python
- Stars
- 2.1k
- Forks
- 612
- Avg merge
- 3d 14h
- Merged PRs (30d)
- 267
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from NVIDIA-NeMo/RL
-
bbox_giou_reward is unbounded: reversed-corner or NaN boxes score far above 1 (RefCOCO env)Possibly taken @jimmy-holdout claimed this 3 days ago. Openbug community-request
Difficulty 2/5 1-3 hours Newbie friendliness 73/100
NVIDIA-NeMo/RL#4871 ·
Maintainers usually reply within 2 days
-
Use Gym’s exported server-type keysPossibly taken @kwistzzqq-byte claimed this today. Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
NVIDIA-NeMo/RL#4800 ·
Maintainers usually reply within 2 days
-
r0.8.0: pyrefly rejects captured optional config in make_policy_factory, blocking CIPossibly taken A pull request linked to this issue is open or already merged. Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
NVIDIA-NeMo/RL#4599 ·
Maintainers usually reply within 2 days
-
Feature
Difficulty 2/5 Half a day Newbie friendliness 78/100
NVIDIA-NeMo/RL#4016 ·
Maintainers usually reply within 2 days
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
NVIDIA-NeMo/RL#2948 ·
Maintainers usually reply within 2 days
Similar issues
-
request-theme
Difficulty 1/5 Under an hour Newbie friendliness 66/100
LizardByte/ThemerrDB#9575 ·
Maintainers usually reply within 1 day
-
author-page correction
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
acl-org/acl-anthology#10599 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
zilliztech/memsearch#818 ·
Maintainers usually reply within 1 day
-
[FAQ] Commit ID for ProjectsPossibly taken @github-actions claimed this today. Openfaq-proposal
Difficulty 1/5 Under an hour Newbie friendliness 84/100
DataTalksClub/faq#503 · 1 comment ·
Maintainers usually reply within 2 days
-
README.zh-CN/ja/es still hard-code 501 test counts after README.md went count-freePossibly taken @Aaqibhafeezkhan claimed this today. Openbeginner documentation good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
BistaDinesh03/critique#164 · 1 comment ·
Maintainers usually reply within 1 day