Hacktoberfest 2026 : les issues que les mainteneurs ont marquées pour octobre, ouvertes et accessibles aux débutants. Parcourir les issues Hacktoberfest

`nemo_speech_tts_synthesize_tokens` ignores `options.voice_name`: every pre-tokenized request is spoken by the default speaker

Ouverte
  • Quelque chose ne va pas ?
#64 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 1 jour

Personne n'a encore pris cette issue.

Évaluation

Difficulté
3/5
Temps estimé
Une demi-journée
Accessibilité débutants
66/100
Type d'issue
Bug
Clarté
Clairement spécifiée
Activité
Active
Stack technique
cpp
Domaine
api, backend

Piste de recherche

Start in src/tts/c_api.cpp by comparing nemo_speech_tts_synthesize_text with nemo_speech_tts_synthesize_tokens, since only the text path passes voice_of(options) through. Then read Synthesizer::synthesize_tokens and prepare() in src/tts/synthesizer.cpp to see where speaker < 0 falls back to default_speaker, and check the voice_name comment in tts.h. Done means the token call with voice_name set and speaker at -1 produces the same PCM byte length as the text call for that voice (180224 in the report), or tts.h is updated to say the token path ignores the voice name.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

Version: NeMo-Speech.cpp built from ffa38cb; the same code is on main at 8642eaa
(VERSION 0.2.0). Model nvidia/magpie_tts_multilingual_357m (v2607.f16.gguf), codec
nemo-nano-codec-22khz-1.89kbps-21.5fps. ROCm/HIP build, Linux.

What happens: nemo_speech_tts_synthesize_text honours options.voice_name, but
nemo_speech_tts_synthesize_tokens does not. With speaker left at its default (-1) and the
voice given only by voice_name, as tts.h allows ("voice_name — ignored when speaker >= 0"),
the token call always speaks with the synthesizer's default speaker (index 0). No error is
returned.

Seen in production: we send pre-tokenized input to mix English phonemes into Italian sentences.
A request for a male speaker came out in the female speaker 0's voice for every sentence that
went through the token call, and in the requested voice for every other sentence.

Repro: same Italian sentence, seed = 1234, language_code = "it-IT", three runs per row.
The byte length is the same in every run; median F0 by pYIN is within 5 Hz:

call voice_name speaker PCM bytes median F0
synthesize_text "2" -1 180224 110 Hz
synthesize_tokens "2" -1 126976 173 Hz
synthesize_tokens "0" -1 126976 173 Hz
synthesize_text "0" -1 126976 173 Hz
synthesize_tokens "" 2 180224 110 Hz

The token call with voice_name = "2" matches speaker 0 exactly. With speaker = 2 set
explicitly it matches the text call for speaker 2.

Cause: in src/tts/c_api.cpp, nemo_speech_tts_synthesize_text copies voice_of(options)
into request.voice_name, and Synthesizer::prepare resolves it. nemo_speech_tts_synthesize_tokens
passes only to_options(options), and Synthesizer::synthesize_tokens (src/tts/synthesizer.cpp)
falls back to impl_->default_speaker when options.speaker < 0. The voice name never reaches
it, and language_code does not either.

Expected: the token call resolves voice_name the same way the text call does when
speaker < 0. Or, if that is not intended, tts.h should document that voice_name is ignored
there and that callers must set speaker.

Possible fix (a suggestion, not compiled): give Synthesizer::synthesize_tokens the voice
name and resolve it like prepare() does:

// synthesizer.cpp, Synthesizer::synthesize_tokens(..., const std::string& voice_name)
if (prepared.options.speaker < 0)
    prepared.options.speaker = resolve_speaker(
        voice_name.empty() ? impl_->default_voice_name : voice_name);

and pass voice_of(options) from nemo_speech_tts_synthesize_tokens in c_api.cpp.

Workaround on our side: the caller sets the numeric speaker field as well as voice_name.
Both calls read it.

Langage dominant
C++
Étoiles
181
Forks
42
Merge moyen
1 j 8 h
PR mergées (30 j)
8

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de NVIDIA/NeMo-Speech.cpp

Toutes les issues de NVIDIA/NeMo-Speech.cpp

Issues similaires

Plus d'issues C++

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.