Bug - Train/inference parity: three divergences between ltx-trainer and ltx-core / ltx-pipelines

Aperta
  • Qualcosa non va?
#283 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
18/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Tranquilla
Stack tecnologico
python, pytorch
Ambito
machine-learning

Direzione di ricerca

Start in ltx-trainer's timestep_samplers.py (shift computed from sequence length near lines 77 and 122) and compare with LTX2Scheduler.execute() in ltx-core, where the shift applies only when latent= is passed. Then check flexible.py around lines 432-440 for the Modality construction without keyframes_mask, and 665-680 for token ordering. Done means a clear decision on which side is intended; the issue is a design question, not a patch, and the maintainers have not yet weighed in.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Version: v1.2.0, commit fd4ded7 (2026-08-11) · Model: LTX 2.5 22B (dev and distilled)

Summary. While reading the repo to implement a custom sequence-concat conditioning scheme, I found three places where the shipped trainer and the shipped inference stack disagree. Two are internal to ltx-trainer alone — its training path versus its own validation runner. None raise an error; all three would silently train a model that behaves differently at sampling time.

  1. σ schedule. ShiftedLogitNormalTimestepSampler derives its shift from actual sequence length (timestep_samplers.py:77,122). At sampling time LTX2Scheduler.execute() only applies the equivalent shift when passed latent= — and of eight call sites, exactly one does (ti2vid_two_stages_hq.py:267). The other seven, including the trainer's own validation_runner.py:875, get the fixed 4096-token default. At the trainer's default validation geometry (960×544×89 → 6120 tokens) training draws σ at mu ≈ 2.77 while validation samples at mu = 2.05.

  2. keyframes_mask. Never set in training — zero grep hits across all five strategy files, Modality built without it at flexible.py:432-440. Always set at inference (tools.py:184, unconditional). Also set by the trainer's own validator (validation_runner.py:783,809). Code default for use_keyframes_abs_pos_embedding is False, which would make this inert — but both LTX 2.5 checkpoints ship it True (read from safetensors metadata).

  3. Token order. Training prepends (flexible.py:665) and slices the tail (:680); inference appends (reference_video_cond.py:97) and every consumer slices the head (clear_conditioning, _first_frame_keyframes_mask, build_attention_mask, multi-GPU padding). Equivalent for the model — attention is dense, positions ride on RoPE — but opposite conventions in the two halves, so porting logic between them fails silently rather than on shape.

What I'm not claiming: I haven't verified how the released checkpoints were actually trained (pretraining code isn't public, so internal training may have matched inference and only the shipped fine-tuner diverged); I haven't measured quality impact; and #3 may be deliberate.

Lingua principale
Python
Stelle
10k
Fork
1.6k
Nessuna PR unita negli ultimi 30g

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di Lightricks/LTX-2

Tutte le issue di Lightricks/LTX-2

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.