Bug - Train/inference parity: three divergences between ltx-trainer and ltx-core / ltx-pipelines

Abierto
  • ¿Algo no está bien?
#283 1 comentario 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
18/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Tranquilo
Stack tecnológico
python, pytorch
Área
machine-learning

Línea de trabajo

Start in ltx-trainer's timestep_samplers.py (shift computed from sequence length near lines 77 and 122) and compare with LTX2Scheduler.execute() in ltx-core, where the shift applies only when latent= is passed. Then check flexible.py around lines 432-440 for the Modality construction without keyframes_mask, and 665-680 for token ordering. Done means a clear decision on which side is intended; the issue is a design question, not a patch, and the maintainers have not yet weighed in.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Version: v1.2.0, commit fd4ded7 (2026-08-11) · Model: LTX 2.5 22B (dev and distilled)

Summary. While reading the repo to implement a custom sequence-concat conditioning scheme, I found three places where the shipped trainer and the shipped inference stack disagree. Two are internal to ltx-trainer alone — its training path versus its own validation runner. None raise an error; all three would silently train a model that behaves differently at sampling time.

  1. σ schedule. ShiftedLogitNormalTimestepSampler derives its shift from actual sequence length (timestep_samplers.py:77,122). At sampling time LTX2Scheduler.execute() only applies the equivalent shift when passed latent= — and of eight call sites, exactly one does (ti2vid_two_stages_hq.py:267). The other seven, including the trainer's own validation_runner.py:875, get the fixed 4096-token default. At the trainer's default validation geometry (960×544×89 → 6120 tokens) training draws σ at mu ≈ 2.77 while validation samples at mu = 2.05.

  2. keyframes_mask. Never set in training — zero grep hits across all five strategy files, Modality built without it at flexible.py:432-440. Always set at inference (tools.py:184, unconditional). Also set by the trainer's own validator (validation_runner.py:783,809). Code default for use_keyframes_abs_pos_embedding is False, which would make this inert — but both LTX 2.5 checkpoints ship it True (read from safetensors metadata).

  3. Token order. Training prepends (flexible.py:665) and slices the tail (:680); inference appends (reference_video_cond.py:97) and every consumer slices the head (clear_conditioning, _first_frame_keyframes_mask, build_attention_mask, multi-GPU padding). Equivalent for the model — attention is dense, positions ride on RoPE — but opposite conventions in the two halves, so porting logic between them fails silently rather than on shape.

What I'm not claiming: I haven't verified how the released checkpoints were actually trained (pretraining code isn't public, so internal training may have matched inference and only the shipped fine-tuner diverged); I haven't measured quality impact; and #3 may be deliberate.

Lenguaje dominante
Python
Estrellas
10k
Forks
1.6k
Sin PR fusionados en 30 d

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de Lightricks/LTX-2

Todos los issues de Lightricks/LTX-2

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.