vllm-project/vllm-omni

[Feature]: Support request-level batching for Wan2.2 pipelines

Fermée

#5 649 ouverte le 31 juil. 2026

 (3 commentaires) (1 réaction) (1 personne assignée)Python (1 067 forks)github user discovery
diffusionenhancementhelp wanted

Métriques du dépôt

Stars
 (4 990 étoiles)
Métriques de merge PR
 (Métriques PR en attente)

Description

🚀 The feature, motivation and pitch

Currently, Wan2.2 pipelines (Wan22Pipeline, Wan22I2VPipeline, Wan22VACEPipeline, Wan22S2VPipeline) do not declare support for request-level batching (supports_request_batch = False). When running offline or online inference with multiple requests in a batch, calling execute_model_batch raises:

RuntimeError: Wan22Pipeline does not support request-batch forward.

Supporting request-level batching (supports_request_batch = True) for Wan2.2 will allow batching multiple prompts/requests in offline batch processing and increase GPU utilization when processing concurrent requests of the same resolution and step count (similar to QwenImagePipeline, SD3Pipeline, FLUXPipeline, and LTX23Pipeline).

Proposed changes:

  1. Declare supports_request_batch = True in Wan2.2 pipeline classes (Wan22Pipeline, Wan22I2VPipeline, etc.).
  2. Update forward(self, req: DiffusionRequestBatch) to accept len(req.prompts) >= 1, removing the single-prompt validation check.
  3. Update encode_prompt() and prepare_latents() to support batched prompt lists and stacked latents ($B > 1$).
  4. Pass batched inputs through predict_noise_maybe_with_cfg() and scheduler.step().

Alternatives

  • Single-request execution (Current): Requests are processed sequentially (execute_request), leading to GPU under-utilization during offline inference benchmarks or multi-prompt batch jobs.
  • Step-wise execution: Requires implementing the full SupportsStepExecution state protocol (prepare_encode, denoise_step, step_scheduler, post_decode), which involves a larger architectural refactor. Request-level batching provides a simpler and faster path to multi-request inference.

Additional context

  • Reference implementations in vLLM-Omni: LTX23Pipeline (vllm_omni/diffusion/models/ltx2/pipeline_ltx2_3.py) and QwenImagePipeline (vllm_omni/diffusion/models/qwen_image/pipeline_qwen_image.py).
  • Pipelines affected: Wan22Pipeline (vllm_omni/diffusion/models/wan2_2/pipeline_wan2_2.py), Wan22I2VPipeline (vllm_omni/diffusion/models/wan2_2/pipeline_wan2_2_i2v.py), Wan22VACEPipeline (vllm_omni/diffusion/models/wan2_2/pipeline_wan2_2_vace.py), and Wan22S2VPipeline (vllm_omni/diffusion/models/wan2_2/pipeline_wan2_2_s2v.py).

Guide contributeur