[Accuracy] PersonaPlex FP16 depth input skips the reference's Half rounding

Cerrado
  • ¿Algo no está bien?
#1,788 0 comentarios 0 reacciones 1 asignado Ver en GitHub

Los mantenedores suelen responder en 1 día

@chaofengw-nv ya está trabajando en esto.

Desde el 18/10/2026.

  • #1751 de @chaofengw-nv — fusionado
  • #1789 de @chaofengw-nv — cerrado sin fusionar

Evaluación

Este issue todavía no se ha evaluado.

Descripción

Summary

When the PersonaPlex depth transformer runs in FP16, the runtime builds its input differently from the reference.

  • Reference: computes depformer_in[cb](hidden) as a Half F.linear, rounds that output to Half, and then adds the
    Half previous-token embedding (depformer_text_emb / depformer_emb) with a Half add.
  • Runtime: prepare_depth_embedding_kernel (families/personaplex/runtime/sampling_kernels.cu) works on FP32 copies
    of the BF16 checkpoint tables. It sums the projection and the embedding in FP32, and the depth engine rounds the
    result to Half only once.

So the FP16 depth path skips the reference's Half rounding of the weights, of the projection output and of the sum.

Minimal counterexample

The inputs are hidden [1, 1] (Half), projection row [1, 2^-12] and embedding -1. All of these values are exact in
both BF16 and Half.

  • Reference: Half(F.linear) = 1, then 1 + (-1) = 0.
  • Current helper: Half(1 + 2^-12 - 1) = 2^-12.

The result does not depend on the reduction order.

Evidence on real data

These results come from GB300 with TensorRT 11.2, using the product kernel compiled from main 88ceb9f52.

  • Counterexample: the product kernel returns 2^-12 for both codebook 0 (text seed) and codebook 1 (audio seed),
    while the reference returns 0.
  • Native FP16 clip-7 capture: steps 0-39, all 16 codebooks, 640 depth steps.
    • The kernel received the reference's own temporal hidden state and previous token. The token was recovered uniquely
      from the reference data in all 640 steps.
    • Its Half depth input differs from the reference depformer input in every one of the 640 steps: 147,457 of 655,360
      elements, about 230 per step.
    • torch.nn.functional.linear in Half on the same GPU reproduces every reference projection bitwise.

Impact

The depth input changes in every FP16 depth step. Near-tie codebook decisions can then flip, and the generated audio
diverges. FP32 depth builds are not affected.

Status

This is a component-level defect: the fix makes the depth input match the reference at its Half boundaries. With the fix
alone, the full-model PersonaPlex audio-parity gate still fails on clip 7, so other differences remain and are tracked
separately.

Lenguaje dominante
Python
Estrellas
288
Forks
73
Merge medio
1 d 7 h
PR fusionados (30 d)
275

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de NVIDIA/TensorRT-Model-Connect

Todos los issues de NVIDIA/TensorRT-Model-Connect

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.