kohya-ss/sd-scripts
Shouldn't fix_noise_scheduler_betas_for_zero_terminal_snr be applied BEFORE prepare_scheduler_for_custom_training?
オープン
#1,905 opened on 2025/01/27
help wanted
Repository metrics
- Stars
- (7,198 個のスター)
- PR merge metrics
- (平均マージ 18h 39m) (30d で 16 merged PRs)
説明
Hello,
Reference code: https://github.com/kohya-ss/sd-scripts/blob/0778dd9b1df0d6aa33287ded3ce4195f3d03251b/train_network.py#L195
I noticed that fix_noise_scheduler_betas_for_zero_terminal_snr modifies the alphas_cumprod for the noise scheduler, and prepare_scheduler_for_custom_training uses alphas_cumprod to calculate all_snr. The issue with the current order is that all_snr will reflect NON-ztsnr snrs, not ztsnr, causing anything that uses all_snr to use snrs that do not align with ztsnr.
As such, shouldn't fix_noise_scheduler_betas_for_zero_terminal_snr always be applied before prepare_scheduler_for_custom_training?
Thank you.