Magpie TTS 2602 and 2607 for Japanese language inserts hallucinated sounds on punctuation symbols (periods).
Maintainers usually reply within 3 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 48/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Active
- Tech stack
- cpp
- Domain
- audio-video-rtc
Research direction
Start by reproducing Japanese Magpie TTS from a source build with Japanese tokenisation enabled, using CJK periods in positions described in the report and comparing commas, Western periods, and final periods. No source files or tests are named; done means punctuation produces pauses without hallucinated sounds across the reported cases.
Written by the indexing model from the issue text.
Description
There are odd sounds inserted where there should be pauses representing CJK periods. It happens for most, but not all periods. The hallucinated sounds to my ear seem to be either mangled Japanese sounds similar to the sounds nearby or seemingly random English-like sounds.
Few things I noticed:
- CJK commas don't cause hallucinations.
- Replacing CJK period with Western period didn't help.
- Final period doesn't cause an issue.
- If you put the period at the beginning of the text, there's an issue.
For reference:
The builds in the releases do not support tokenisation for Japanese language for Magpie TTS. So I built it from source with Japanese support (in my case Windows with CPU backend).
- Dominant language
- C++
- Stars
- 167
- Forks
- 37
- Avg merge
- 7d 17h
- Merged PRs (30d)
- 8
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from NVIDIA/NeMo-Speech.cpp
-
Difficulty 4/5 3-5 days Newbie friendliness 52/100
NVIDIA/NeMo-Speech.cpp#61 ·
Maintainers usually reply within 3 days
-
enhancement
NVIDIA/NeMo-Speech.cpp#56 · 3 comments ·
Maintainers usually reply within 3 days
-
Expose confidence estimation / token probabilities (currently always 1.0)Possibly taken @pskrunner14 claimed this 8 days ago. Openenhancement
Difficulty 4/5 3-5 days Newbie friendliness 48/100
NVIDIA/NeMo-Speech.cpp#55 · 2 comments · 1 assignee ·
Maintainers usually reply within 3 days
-
Difficulty 4/5 3-5 days Newbie friendliness 52/100
NVIDIA/NeMo-Speech.cpp#49 · 1 comment · 1 reaction ·
Maintainers usually reply within 3 days
-
Streaming RNNT wedges after sustained zero-PCM silence; Vulkan aborts with GGML_ASSERT(ne3 == ne13)Open
Difficulty 4/5 3-5 days Newbie friendliness 48/100
NVIDIA/NeMo-Speech.cpp#48 ·
Maintainers usually reply within 3 days
All issues in NVIDIA/NeMo-Speech.cpp
Similar issues
-
agent:Windows bug
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
? - Needs Triage bot_watch bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
NVIDIA/cudf-spark-jni#5267 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Maintainers usually reply within 1 day
-
(S1 - Need confirmation)
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
CleverRaven/Cataclysm-DDA#88974 ·
Maintainers usually reply within 1 day