Running out of memory with version v0.5.17
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 25/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- python
- Domain
- machine-learning
Research direction
Reproduce the report on an A100 with v0.5.17, first loading Qwen2.5-7B and then comparing the same setup against v0.5.6. Investigate the memory difference around model loading and the newer unsloth dependency. Done means identifying the regression or missing configuration needed to restore the earlier memory usage.
Written by the indexing model from the issue text.
Description
I used v0.5.6 but upgraded to version v0.5.17 since it uses the newer version of unsloth, which is supposed to be much more memory efficient, especially for MOE models and RL. But with the newer version, I get an out-of-memory error for the same model and the same setup with which v0.5.6 worked fine. I'm training Qwen2.5-14B on A100 GPU. Now, even if I try to run Qwen2.5-7B I get out of memory while loading the model, not even when training starts.
- Dominant language
- Python
- Stars
- 10.8k
- Forks
- 997
- Avg merge
- 10h 1m
- Merged PRs (30d)
- 117
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from OpenPipe/ART
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 54/100
OpenPipe/ART#961 · 3 comments ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
OpenPipe/ART#949 · 5 comments ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 10/100
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 42/100
Maintainers usually reply within 1 day
Similar issues
-
good first issue hacktoberfest
Difficulty 1/5 Under an hour Newbie friendliness 92/100
RogueAlg0/taken#387 · 4 comments ·
Maintainers usually reply within 1 day
-
tool-calling
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
vllm-project/vllm#59838 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 92/100
raullenchai/Rapid-MLX#4042 ·
Maintainers usually reply within 1 day
-
documentation
Difficulty 1/5 Under an hour Newbie friendliness 92/100
transitmatters/mbta-slow-zone-bot#70 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
open-webui/open-webui#31871 · 1 comment ·
Maintainers usually reply within 1 day