[DDIMInverseScheduler] `inf` values at first iteration when `set_alpha_to_one=True` and `prediction_type="sample"`

Open Beginner friendly
#10,920 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
76/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
python, pytorch
Domain
machine-learning

Research direction

Start in src/diffusers/schedulers/scheduling_ddim_inverse.py around lines 338 and 347, where alpha and beta products are computed during the first step. Run the provided DDIMInverseScheduler reproduction with prediction_type="sample" and set_alpha_to_one=True, then verify the first step produces no inf values.

Written by the indexing model from the issue text.

Description

bug stale
Describe the bug

I got inf values in https://github.com/huggingface/diffusers/blob/560fb5f4d65b8593c13e4be50a59b1fd9c2d9992/src/diffusers/schedulers/scheduling_ddim_inverse.py#L347

because beta_prod_t is 0.0 at the first iteration, when timestep is < 0 and beta_prod_t is 0.0 (because alpha_prod_t is set to 1.0 when set_alpha_to_one=True).

https://github.com/huggingface/diffusers/blob/560fb5f4d65b8593c13e4be50a59b1fd9c2d9992/src/diffusers/schedulers/scheduling_ddim_inverse.py#L338

Reproduction
import torch as th 
from diffusers import DDIMInverseScheduler

ddim_inverse_scheduler = DDIMInverseScheduler(
    num_train_timesteps=1000,
    prediction_type="sample",
    set_alpha_to_one=True
)
ddim_inverse_scheduler.set_timesteps(num_inference_steps=50)

with th.no_grad():
    pred = th.randn((1, 1, 2, 2, 2))
    samples = th.randn((1, 1, 2, 2, 2))
    t = 0
    samples = ddim_inverse_scheduler.step(pred, t, samples).prev_sample
    assert not th.isinf(samples).any(), "samples contain inf values"
Logs
System Info
  • 🤗 Diffusers version: 0.32.2
  • Platform: macOS-15.2-arm64-arm-64bit
  • Running on Google Colab?: No
  • Python version: 3.10.8
  • PyTorch version (GPU?): 2.5.1 (False)
  • Flax version (CPU?/GPU?/TPU?): not installed (NA)
  • Jax version: not installed
  • JaxLib version: not installed
  • Huggingface_hub version: 0.29.1
  • Transformers version: not installed
  • Accelerate version: not installed
  • PEFT version: not installed
  • Bitsandbytes version: not installed
  • Safetensors version: 0.5.2
  • xFormers version: not installed
  • Accelerator: Apple M1 Pro
  • Using GPU in script?: No (Using MPS)
  • Using distributed or parallel set-up in script?: No
Who can help?

@yiyixuxu @sayakpaul

Dominant language
Python
Stars
34.6k
Forks
7.3k
Avg merge
3d 16h
Merged PRs (30d)
74

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from huggingface/diffusers

All issues in huggingface/diffusers

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.