Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Converter caps MagpieTTS and NanoCodec at f16/f32 while ASR defaults to q8_0

Cerrado
  • ¿Algo no está bien?
#8 1 comentario 0 reacciones 1 asignado Ver en GitHub

Los mantenedores suelen responder en 2 días

@anand-nv ya está trabajando en esto.

Desde el 10/8/2026.

Evaluación

Este issue todavía no se ha evaluado.

Descripción

Summary

conversion/registry.py allows eight weight types for ASR and two for TTS and codec. I would like to understand whether that gap is a measured quality decision or an unimplemented path, and I am proposing a specific experiment that would settle it either way.

This follows a LinkedIn thread where I was asked to bring the question here.

What the code does today

_normalized_outtype in conversion/registry.py sets the defaults and the allowed sets:

architecture default supported
asr q8_0 f16, bf16, q8_0, q4_k, q5_k, q6_k, nvfp4, mxfp4
pnc q8_0 f16, bf16, q8_0
diarization f32 f32, f16, bf16, q8_0, q4_k, q5_k, q6_k, nvfp4, mxfp4
tts f16 f16, f32
codec f16 f16, f32

So python3 convert_model.py nvidia/magpie_tts_multilingual_357m --outfile m.gguf --outtype q8_0 raises ValueError, and the same holds for the NanoCodec checkpoint.

The restriction appears to track the converter rather than a quality finding. conversion/tts.py does not import conversion/quantization.py, and tensor_to_numpy only ever emits torch.float16 or torch.float32. conversion/codec.py is in the same position. There is no code path that could produce a quantized TTS tensor, so as far as I can tell no q8_0 MagpieTTS GGUF has been measured.

Why the TTS side looks like a candidate

Three things in the existing converter already point the right way.

should_store_f32 in conversion/tts.py keeps rank-1 tensors and every .bias at f32 while letting rank-2 weights drop to f16. That is the same keep-the-norms-high convention a q8_0 path needs, already written and already applied.

add_tensors splits the Conv1d weights *.pos_ff.proj.conv.weight and *.pos_ff.o_net.conv.weight into per-kernel rank-2 slices .k0 through .kN. Those land in ggml_mul_mat through src/runtime/ggml/nn.cpp, which is the same call that makes Q8_0 and the k-quants compute natively on the ASR side. The stored layout is already the shape a quantized matmul consumes.

--local-transformer-outtype already exists and already carries a per-subgraph precision override, with is_local_transformer_tensor scoping it to audio_embeddings., local_transformer., local_transformer_in_projection. and local_transformer_out_projections.. Widening its choices is a smaller change than introducing the concept.

Finally, MagpieTTS emits discrete NanoCodec tokens rather than a waveform or a mel, so a weight perturbation either changes the selected token or it does not survive to the output at all.

The counter-argument I cannot dismiss

That last point is weaker than it first looks, and it is the reason I am opening a question rather than a request.

Decoding is not argmax. The converter writes magpietts.inference.topk at 80, magpietts.inference.temperature at 0.6 and magpietts.inference.cfg_scale at 2.5. Under top-k sampling with classifier-free guidance, small logit shifts do change the sampled token, and errors accumulate over up to max_decoder_steps autoregressive frames with the attention prior reacting to the drift. NanoCodec at 21.5 fps also has no downstream stage that rounds an error away, which cuts against quantization here in a way it does not for an ASR decoder.

So the argument that discrete output makes 8-bit cheap holds for a greedy decoder and is not established for this one.

Proposed test

Fixed seed, a fixed prompt set across the supported languages, both models converted from the same checkpoint, comparing at the token level before anything reaches the codec:

  1. Convert nvidia/magpie_tts_multilingual_357m at f16 and at q8_0.
  2. Generate with identical seed, text, speaker and sampling parameters.
  3. Compare the emitted NanoCodec token streams frame by frame, and record exact-match rate, first divergence frame and generated length.
  4. Repeat at temperature=0 to separate weight error from sampling noise.

Two outcomes, both useful. If the token streams agree at greedy decode and diverge only marginally under sampling, f16 on the TTS side is over-provisioning and the ASR precedent carries, so a q8_0 path is worth adding. If they diverge materially at greedy decode, the current default is correct and the interesting artifact is the number, which belongs in docs/model-conversion.md next to the table so the question stops coming up.

A --local-transformer-outtype q8_0 variant is worth a row too, since the local transformer and audio embeddings are the cheapest subgraph to move and the plumbing is already there.

Questions
  1. Has a quantized MagpieTTS or NanoCodec conversion been measured internally, and if so is the number publishable?
  2. Is the {f16, f32} restriction a deliberate quality floor, or is it recording that the converter has no quantization path yet?
  3. If the experiment above lands on the favorable side, would a q8_0 path in conversion/tts.py plus the matching entry in tests/conversion/converter_contract_test.py be in scope for a PR?

I am happy to run the experiment and post the table here before writing any converter code.

Environment

Observations are from the default branch at 5be7bfb104802131e61fe679b3f1401b27270216.

Lenguaje dominante
C++
Estrellas
167
Forks
37
Merge medio
6 d 23 h
PR fusionados (30 d)
9

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de NVIDIA/NeMo-Speech.cpp

Todos los issues de NVIDIA/NeMo-Speech.cpp

Issues similares

Más issues de C++

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.