[BUG in Stable Diffusion inference] There's an error on CUDAGraph when using deepspeed inference. How to fix it?

Open

#866 opened on Feb 29, 2024 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty: 4/5
Estimated time: 3-5 days
Newbie friendliness: 35/100

Issue type: Bug
Clarity: Needs clarification
Activity status: Stale
Tech stack: python, pytorch
Domain: machine-learning

Research direction

Start with the Stable-Diffusion inference example linked in the issue, especially test-stable-diffusion.py, and reproduce it with the supplied deepspeed command and environment versions. Inspect the linked traceback to determine why CUDAGraph is unavailable during inference; done means identifying a reproducible cause and a verified fix or compatible configuration.

Written by the indexing model from the issue text.

Description

【Repo Link】:
Stable-Diffusion inference

【The Command used to run】:
deepspeed --num_gpus 1 test-stable-diffusion.py

【Envs】:

RTX 3090
deepspeed 0.12.6
torch 1.13.1
diffusers 0.26.1
triton 2.0.0.dev20221202

【Traceback information】:

It seems like the CUDAGraph is not available, which confused me a lot.
Please take time to review the issue and give me some feedback. Appreciate!

Dominant language: Python
Stars: 6.8k
Forks: 1.1k
Avg merge: 2d 16h
Merged PRs (30d): 1

Contributor guide

No contributing guide indexed for this repository

First steps

Read the whole issue, then the project's contributing guide.
Comment on the issue to say you are picking it up — it saves two people doing the same work.
Fork the repository and make your change on a branch.
Open a pull request that references the issue number.

Similar issues

slack-gif-creator: validate_gif() multiplies the last frame's duration by the frame count, so reported duration/fps are wrong (1.0s GIF reported as 1.8s) Open

Difficulty 2/5 1-3 hours Newbie friendliness 75/100

anthropics/skills#1811 · 1 comment · Opened Sep 23, 2026
App fails to start with gradio 6 (allowed by `gradio>=5.49.1`) Open

Difficulty 2/5 1-3 hours Newbie friendliness 75/100

speaches-ai/speaches#678 · Opened Sep 23, 2026
ModuleNotFoundError: No module named 'mcp.server.fastmcp'. Open

bug

Difficulty 2/5 1-3 hours Newbie friendliness 75/100

datalayer/mcp-compose#42 · Opened Sep 23, 2026
import spacy fails with ModuleNotFoundError: No module named 'click' Open

Difficulty 2/5 1-3 hours Newbie friendliness 75/100

conda-forge/spacy-feedstock#177 · Opened Sep 23, 2026
humaneval: extract_function_body corrupts a fenced body-only reply, so the same correct code scores INCORRECT fenced and CORRECT unfenced Open

Difficulty 2/5 1-3 hours Newbie friendliness 70/100

UKGovernmentBEIS/inspect_evals#2523 · Opened Sep 23, 2026

More Python issues