yl4579/StyleTTS2

Fine-tuning or training from scratch in a differente language?

Open

#197 opened on Jan 30, 2024

View on GitHub
 (30 comments) (0 reactions) (0 assignees)Python (210 forks)batch import
help wanted

Repository metrics

Stars
 (3,429 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

Hi everyone, I'm considering putting some effort into training StyleTTS in Portuguese. I have a good-quality dataset for this task, however, I was in doubt if it would be better just to fine-tune the model (which I know was trained in English), or (since it's an unseen language) train the model from scratch in Portuguese.

Does anyone have some tips on what I should consider before making a decision?

Contributor guide

Research direction
Compare the size and quality of your Portuguese dataset with the original English dataset. Consider fine tuning if your dataset is small (e.g., <10 hours) to leverage pre trained English features, or train from scratch if you have a large dataset (e.g., >100 hours) to capture language specific prosody. Also evaluate computational resources: fine tuning is cheaper.
Tech stack
python
Domain
machine learningaiaudio video rtc
Issue type
Research
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
PythonMachine Learning
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
70