SDXL-1 to ONNX conversion
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 15/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- python, pytorch
- Domain
- machine-learning
Research direction
Start by reproducing the provided SDXL UNet export and ONNX Runtime quantization code, then inspect why the exported model is 3.98 MB while the quantized model is much larger and why inference loading fails. Done would require a valid quantized SDXL UNet model that loads successfully for inference, but the issue does not name a repository file or test to update.
Written by the indexing model from the issue text.
Description
Hi i am facing issue in Quantization on SDXL 1 when i export UNet component to .ONNX model and the i am not able to create .ONNX Model
when i export using the code given below it export the unet.ONNX model of 4mb which is too small
and also facing an issue while loading it for inference
actually i want to quantized the model and used for edge devices for image to image inference to save memory and time could you guide me completely to Quantized the SDXL-1
!pip install -q diffusers transformers accelerate onnx onnxruntime
from diffusers import UNet2DConditionModel
import torch
Load the UNet model from SDXL
unet = UNet2DConditionModel.from_pretrained(
"stabilityai/stable-diffusion-xl-base-1.0",
subfolder="unet",
torch_dtype=torch.float32 # use float32 for export
)
unet.eval()
import torch.nn as nn
class UNetWrapper(nn.Module):
def init(self, unet):
super().init()
self.unet = unet
def forward(self, sample, timestep, encoder_hidden_states, text_embeds, time_ids):
added_cond_kwargs = {
"text_embeds": text_embeds,
"time_ids": time_ids
}
return self.unet(
sample=sample,
timestep=timestep,
encoder_hidden_states=encoder_hidden_states,
added_cond_kwargs=added_cond_kwargs
).sample
sample = torch.randn(1, 4, 64, 64) # latent input
timestep = torch.tensor([10], dtype=torch.int64) # denoising step
encoder_hidden_states = torch.randn(1, 77, 2048) # text encoder output
text_embeds = torch.randn(1, 1280) # pooled text embeddings
time_ids = torch.randn(1, 6) # time conditioning vector
from pathlib import Path
Define output path
onnx_path = "/content/drive/MyDrive/Umair/SkinCare/Quantization/Dynamic Quantization unet/unet_sdxl.onnx"
Path(onnx_path).parent.mkdir(parents=True, exist_ok=True)
Export the model
torch.onnx.export(
UNetWrapper(unet),
(sample, timestep, encoder_hidden_states, text_embeds, time_ids),
onnx_path,
input_names=["sample", "timestep", "encoder_hidden_states", "text_embeds", "time_ids"],
output_names=["out_sample"],
dynamic_axes={
"sample": {0: "batch"},
"encoder_hidden_states": {0: "batch"},
"text_embeds": {0: "batch"},
"time_ids": {0: "batch"},
"out_sample": {0: "batch"},
},
opset=17
)
print("✅ UNet exported successfully to:", onnx_path)
from onnxruntime.quantization import quantize_dynamic, QuantType
quantized_path = onnx_path.replace(".onnx", "_int8.onnx")
quantize_dynamic(
model_input=onnx_path,
model_output=quantized_path,
weight_type=QuantType.QInt8
)
print("✅ Quantized model saved to:", quantized_path)
code to check the size of .onnx and _int8.onnx
print(f".onnx file size: {os.path.getsize(onnx_model_path) / (10241024):.2f} MB")
print(f"_int8.onnx file size: {os.path.getsize(quant_model_path) / (10241024):.2f} MB")
.onnx file size: 3.98 MB
_int8.onnx file size: 2458.93 MB
and also facing an issue while loading it for infrence
- Dominant language
- C++
- Stars
- 1.7k
- Forks
- 417
- PR merge metrics
- No merged PRs in 30d
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- No Dockerfile or Docker Compose file
- No pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from microsoft/onnxruntime-inference-examples
-
Replace deprecated light-server with lite-server for onnxruntime-webPossibly taken A pull request linked to this issue is open or already merged. Open
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Minor: typoPossibly taken @Hashim1999164 claimed this 66 days ago. Open
Difficulty 1/5 Under an hour Newbie friendliness 72/100
microsoft/onnxruntime-inference-examples#124 · 1 reaction ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 38/100
-
Package "onnxruntime-react-native" does not contain a valid config pluginPossibly taken @reginaldalfret claimed this 24 days ago. Open
Difficulty 3/5 1-2 days Newbie friendliness 42/100
-
Difficulty 3/5 1-2 days Newbie friendliness 45/100
All issues in microsoft/onnxruntime-inference-examples
Similar issues
-
agent:WSL bug code-review linux LOW
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Maintainers usually reply within 1 day
-
category:port-update
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
Maintainers usually reply within 2 days
-
improvement ui
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
ihhub/fheroes2#11062 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
Maintainers usually reply within 1 day
-
enhancement question
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
DarkStarSystems/pcons#224 ·
Maintainers usually reply within 1 day