vllm-project/vllm-omni

[RFC]: Diffusion Offline Examples Docs & Test

Open

#1,244 opened on Feb 6, 2026

View on GitHub
 (27 comments) (1 reaction) (1 assignee)Python (1,067 forks)github user discovery
documentationenhancementgood first issuehelp wanted

Repository metrics

Stars
 (4,990 stars)
PR merge metrics
 (PR metrics pending)

Description

Motivation.

This PR proposes a template for the offline inference example document and respective test script implementation.

🙋: help wanted!

Examples Docs Refinement Test script Assignee PR
examples/offline_inference/text_to_image/README.md tests/examples/offline_inference/test_text_to_image.py @fhfuih #1910
examples/offline_inference/text_to_video/text_to_video.md tests/examples/offline_inference/test_text_to_video.py @jackywangno007-cyber
examples/offline_inference/text_to_audio/README.md tests/examples/offline_inference/test_text_to_audio.py @mistcoversmyeyes
examples/offline_inference/image_to_image/image_to_image.md tests/examples/offline_inference/test_image_to_image.py @wtomin #2417
examples/offline_inference/image_to_video/README.md tests/examples/offline_inference/test_image_to_video.py @loveysuby #3573
examples/offline_inference/text_to_audio/README.md tests/examples/offline_inference/test_text_to_audio.py 🙋

1. Offline Inference Example Document

All .md files under docs/user_guide/examples/ are auto-generated at build time and should not be edited manually. The true sources are the corresponding README.md or standalone .md files in the examples/ directory.

The offline inference documents under docs/user_guide examples/offline_inference/ about diffusion models are five:

  • text_to_audio/*.md
  • image_to_video/*.md
  • image_to_image/*.md
  • text_to_video/*.md
  • text_to_image/*.md

In addtion, there are some independent models offline inference document, for example, examples/offline_inference/bagel/README.md.

Each document has different structure. Some information might be missing. It is better to follow a unified template for these document

2. Offline Inference Example Test

For each task, there is supposed to be one test script that verify the functionality of all the examples in the current example document. For example:

tests/examples/offline_inference/test_text_to_image.py

One related PR is #1910. test_text_to_image.py will automatically extract the code snippets from the respective readme document (examples/offline_inference/text_to_image/README.md), and run them to verify its functionality.

Proposed Change.

I have a suggested template for offline inference example:


# Task Name

Brief introduction. 

## Table of Content

## Overview

Task introduction

### Supported Models

| Model | Image/Video Shape  | Peak VRAM (GiB) | Model Weights (GiB)  |
| `Model/Name-A` | height x width  |  xxx|  xxx|
| `Model/Name-B` | height x width |  xxx  | xxx|

!!! info
    Peak VRAM:  based on basic single-card usage, batch size =1, without any acceleration/optimization features. Some models weights cannot fit into one card with 80GiB VRAM, which may need to use cpu offloading.
    Model Weights: the VRAM consumption of model weights (usually BF16) is printed as `Model loading took xxx GiB and xxx seconds` in the logging.


Default model : `Model/Name-A`

## Prerequisites (Optional)

install special dependencies, needs to login HF to access gated model, etc.
For some latest models, it may require special diffusers version.

## Quick Start
Use the default model

### Python API
For example:

from vllm_omni.entrypoints.omni import Omni
if __name__ == "__main__":
    omni = Omni(model="Qwen/Qwen-Image")
    prompt = "a cup of coffee on the table"
    outputs = omni.generate(prompt)
    images = outputs[0].request_output[0].images
    images[0].save("coffee.png")

### Local CLI Usage
The same example, but in local CLI usage

python text_to_image.py \
  --model  "Qwen/Qwen-Image" \
  --prompt "a cup of coffee on the table" \
  --output coffee.png

##  Key Arguments

A table for parameters, including Argument、Type、Default、Description 

| Argument | Type | Default | Description |
| --------- | ---- | ------- | ------------- | 
| `height` | int | `{default}` | Output image height in pixels |
| `width` | int | `{default}` | Output image width in pixels |

## More CLI Examples

change the model names, change parameters, batch request (optional)

### Advanced Features (Optional)

e.g., CFG-parallel, maybe provide link to `user_guide/diffusion_acceleration.md#supported-models`.

e.g., inference with LoRA weight

## Web UI Demo (Optional)

Gradio demo launching method

## FAQ (Optional)


Feedback Period.

No response

CC List.

No response

Any Other Things.

@hsliuustc0106 @ZJY0516 @SamitHuang @dongbo910220 @lishunyang12 @mxuax @fhfuih

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

Contributor guide