Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

`nemo_speech_tts_synthesize_tokens` ignores `options.voice_name`: every pre-tokenized request is spoken by the default speaker

Abierto
  • ¿Algo no está bien?
#64 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 2 días

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
3/5
Tiempo estimado
Medio día
Aptitud para principiantes
66/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
cpp
Área
api, backend

Línea de trabajo

Start in src/tts/c_api.cpp by comparing nemo_speech_tts_synthesize_text with nemo_speech_tts_synthesize_tokens, since only the text path passes voice_of(options) through. Then read Synthesizer::synthesize_tokens and prepare() in src/tts/synthesizer.cpp to see where speaker < 0 falls back to default_speaker, and check the voice_name comment in tts.h. Done means the token call with voice_name set and speaker at -1 produces the same PCM byte length as the text call for that voice (180224 in the report), or tts.h is updated to say the token path ignores the voice name.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Version: NeMo-Speech.cpp built from ffa38cb; the same code is on main at 8642eaa
(VERSION 0.2.0). Model nvidia/magpie_tts_multilingual_357m (v2607.f16.gguf), codec
nemo-nano-codec-22khz-1.89kbps-21.5fps. ROCm/HIP build, Linux.

What happens: nemo_speech_tts_synthesize_text honours options.voice_name, but
nemo_speech_tts_synthesize_tokens does not. With speaker left at its default (-1) and the
voice given only by voice_name, as tts.h allows ("voice_name — ignored when speaker >= 0"),
the token call always speaks with the synthesizer's default speaker (index 0). No error is
returned.

Seen in production: we send pre-tokenized input to mix English phonemes into Italian sentences.
A request for a male speaker came out in the female speaker 0's voice for every sentence that
went through the token call, and in the requested voice for every other sentence.

Repro: same Italian sentence, seed = 1234, language_code = "it-IT", three runs per row.
The byte length is the same in every run; median F0 by pYIN is within 5 Hz:

call voice_name speaker PCM bytes median F0
synthesize_text "2" -1 180224 110 Hz
synthesize_tokens "2" -1 126976 173 Hz
synthesize_tokens "0" -1 126976 173 Hz
synthesize_text "0" -1 126976 173 Hz
synthesize_tokens "" 2 180224 110 Hz

The token call with voice_name = "2" matches speaker 0 exactly. With speaker = 2 set
explicitly it matches the text call for speaker 2.

Cause: in src/tts/c_api.cpp, nemo_speech_tts_synthesize_text copies voice_of(options)
into request.voice_name, and Synthesizer::prepare resolves it. nemo_speech_tts_synthesize_tokens
passes only to_options(options), and Synthesizer::synthesize_tokens (src/tts/synthesizer.cpp)
falls back to impl_->default_speaker when options.speaker < 0. The voice name never reaches
it, and language_code does not either.

Expected: the token call resolves voice_name the same way the text call does when
speaker < 0. Or, if that is not intended, tts.h should document that voice_name is ignored
there and that callers must set speaker.

Possible fix (a suggestion, not compiled): give Synthesizer::synthesize_tokens the voice
name and resolve it like prepare() does:

// synthesizer.cpp, Synthesizer::synthesize_tokens(..., const std::string& voice_name)
if (prepared.options.speaker < 0)
    prepared.options.speaker = resolve_speaker(
        voice_name.empty() ? impl_->default_voice_name : voice_name);

and pass voice_of(options) from nemo_speech_tts_synthesize_tokens in c_api.cpp.

Workaround on our side: the caller sets the numeric speaker field as well as voice_name.
Both calls read it.

Lenguaje dominante
C++
Estrellas
167
Forks
37
Merge medio
6 d 23 h
PR fusionados (30 d)
9

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de NVIDIA/NeMo-Speech.cpp

Todos los issues de NVIDIA/NeMo-Speech.cpp

Issues similares

Más issues de C++

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.