loss nan in VAE training
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 25/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- python
- Domain
- machine-learning
Research direction
Start by reproducing training from scratch with the stable_audio_2_0_vae.json configuration, using both 44100 Hz mono and stereo audio as described. Inspect the training output around the step where the loss becomes NaN and compare repeated runs. Done means determining whether the behavior is expected and documenting or addressing the cause of the instability.
Written by the indexing model from the issue text.
Description
Thank you for your excellent work and the well-designed open-source code.
When I use your training code to train from scratch, I frequently encounter a situation where the loss becomes NaN after a certain number of training steps. Is this behavior expected?
This issue occurs when training both 44100 mono and stereo audio files. I have to repeat the training multiple times to ensure the loss remains stable.
I am using the stable audio 2.0 config.
- Dominant language
- Python
- Stars
- 3.9k
- Forks
- 488
- PR merge metrics
- No merged PRs in 30d
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from Stability-AI/stable-audio-tools
-
Possible incorrect use of `and` with multiple context managersPossibly taken @rodriguescarson claimed this 123 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Stability-AI/stable-audio-tools#238 · 2 comments ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
-
run_gradio.py fails out of the box on Apple Silicon (MPS): float64 crash, slice TypeError, missing ui depsMay be free again A pull request for this issue was closed without being merged. Open
Difficulty 4/5 3-5 days Newbie friendliness 72/100
-
Difficulty 4/5 3-5 days Newbie friendliness 45/100
-
Difficulty 3/5 1-2 days Newbie friendliness 25/100
All issues in Stability-AI/stable-audio-tools
Similar issues
-
question
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
conda-forge/vc-feedstock#132 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
limitlessprojects/Go-Hybrid#3342 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
UKGovernmentBEIS/inspect_ai#6268 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
LSSTDESC/rail_sompz#42 ·
-
All tests fail with 'EXITED with status 3' on pytest 9.2.0 (TestReport._id is not marshallable)Open
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
pytest-dev/pytest-forked#113 ·