RFC: Deprecate allow_training_without_logprobs option
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 30/100
- Issue type
- Refactor
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- python
- Domain
- machine-learning
Research direction
Start by reviewing the proposal and related PR #527, then locate every use of the allow_training_without_logprobs option in the Python codebase. Confirm whether any users depend on the option and whether its removal is accepted; done means a maintainer decision followed by complete removal of the option and its alternative training path.
Written by the indexing model from the issue text.
Description
Summary
We're considering removing the allow_training_without_logprobs option from ART. This RFC is to gather community feedback before making this change.
Background
The allow_training_without_logprobs option allows training without requiring generation logprobs from the model. However, this approach has several drawbacks:
-
Importance sampling requires logprobs for stable training: In our experiments and in the wider RL community, having generation logprobs is essential for importance sampling, which is critical for stable training results. Training without them leads to less reliable outcomes.
-
Code complexity: Maintaining this alternative path adds complexity to the codebase and makes it harder to reason about the training flow.
-
Subtle bugs: The additional code path creates opportunities for subtle bugs. For example, in PR #527 we discovered tool-call tokenization issues that were partially enabled by this mode's complexity.
Proposal
Remove the allow_training_without_logprobs option entirely, simplifying the codebase and ensuring all users benefit from the more robust training path that uses logprobs.
Request for Feedback
Is anyone in the community actively using allow_training_without_logprobs with good results?
If you're using this option and it's working well for your use case, please let us know:
- What is your use case?
- Why do you need to train without logprobs?
- What results are you seeing?
If we don't hear from users who depend on this feature, we plan to remove it in an upcoming release.
Related: #527
- Dominant language
- Python
- Stars
- 10.8k
- Forks
- 989
- Avg merge
- 11h 53m
- Merged PRs (30d)
- 97
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from OpenPipe/ART
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
-
Difficulty 5/5 Over a week Newbie friendliness 10/100
-
Difficulty 5/5 Over a week Newbie friendliness 42/100
-
enhancement
Difficulty 4/5 3-5 days Newbie friendliness 48/100
Similar issues
-
agent-ready documentation needs-triage
Difficulty 1/5 1-3 hours Newbie friendliness 88/100
-
documentation
Difficulty 1/5 Under an hour Newbie friendliness 91/100
-
workflow-status page template still says reusable workflows are "triggered only by workflow_call:" Open
Difficulty 1/5 Under an hour Newbie friendliness 92/100
-
instance instance add
Difficulty 1/5 Under an hour Newbie friendliness 72/100
searxng/searx-instances#939 · 1 comment ·
-
area-deployment area-integrations triage:bot-seen
Difficulty 2/5 Half a day Newbie friendliness 86/100