RVC-Project/Retrieval-based-Voice-Conversion-WebUI

African Language Phonemes

Open

#2,057 opened on 2024年5月14日

GitHub で見る
 (3 comments) (0 reactions) (0 assignees)Python (18,427 stars) (2,849 forks)batch import
bugdocumentationfollowing uphelp wanted

説明

Hi!

I'm trying to train RVC models for a range of dialects in my country. I live in South Africa.

I'm struggling with click-consonant phonemes.

These African language sounds are not understood as phonemic by the model, so transforming voice-to-voice using recordings of African language speakers results in dropped phonemes or incorrect phonemes.

Click consonants are not reliably preserved in the output and are even occasionally replaced by other consonants ("c", "ck", "k").

Is there anyway for me to overcome this roadblock? Do I need to include more of the click-consonants in my training data?

Click Consonant Transformation.zip

コントリビューターガイド

技術スタック
pythonpytorch
領域
machine learningai
Issue 種別
bug
難度新しい貢献者にとっての実装難度です。1 は非常に小さな変更、5 は専門的な作業です。
4
推定時間経験ある貢献者が調査、実装、テスト、pull request 準備にかけるおおよその時間範囲です。
over 1 week
活動状況Issue が今どれだけ取り組みやすいかを示します。新しい、活発、古い、ブロック中、またはメンテナー入力待ちなどです。
stale
明確さ期待される変更、受け入れ条件、次の手順がどれだけ明確かを示します。
needs investigation
前提条件
phoneticsmachine learning basicsaudio processing
初心者向け度初回貢献者にどれだけ取り組みやすいかを 1-100 で推定したスコアです。
20
調査方針
The issue reports that click consonant phonemes are dropped or replaced during voice conversion. Examine the model's phoneme dictionary and acoustic model implementation, likely in the model/ or voice conversion/ directories. Review the attached zip file with example audio to understand the failure modes. Check if the training data includes click consonants; if not, consider augmenting the dataset. Engage with maintainers on potential modifications to the phoneme set or preprocessing pipeline. See community comments for any suggested workarounds.