facebookresearch/fairseq
Is same base model used for MMS-ASR and MMS-TTS (like AudioPaLM)?
オープン
#5,221 opened on 2023/06/27
enhancementhelp wantedneeds triage
Repository metrics
- Stars
- (29,107 個のスター)
- PR merge metrics
- (PR metrics pending)
説明
Google introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation.
How about the MMS? I found fine-tuning MMS ASR based on pretrained base model mms-1b, but I can not find for TTS. Is the same base model mms-1b used for MMS-TTS? How can I fine-tuning or add new language for TTS?