vllm-project/vllm-omni

[RFC]: Enable NPU CI Tests

Open

#3,565 opened on May 13, 2026

View on GitHub
 (1 comment) (0 reactions) (0 assignees)Python (1,067 forks)github user discovery
NPUhelp wanted

Repository metrics

Stars
 (4,990 stars)
PR merge metrics
 (PR metrics pending)

Description

Nightly Pipeline RFC

Background

This RFC proposes adding and improving nightly test coverage for vLLM-Omni core models.
The goal is to make the nightly pipeline more stable, observable, and useful for catching functionality, performance, and accuracy regressions early.

Pipeline Plan

Core Models Monitor

The following core models should be covered by the nightly monitoring pipeline.
For each model, we should track functionality, performance, and accuracy where applicable.

Model Functionality Test Performance Test Accuracy Test Status Notes
Wan2.2 TBD TBD Planned Initial functionality coverage is being added in PR #3480
Hunyuan-Image 3.0 TBD TBD TBD Planned Image generation model coverage
Qwen-Image TBD TBD TBD Planned Image generation model coverage
Qwen3-TTS ✅ #3600 TBD TBD Planned TTS model coverage
Qwen3-Omni TBD TBD TBD Planned Omni / multimodal model coverage

CI Challenges

Ideally, after a vLLM release, vllm-ascend would promptly publish a corresponding rc1 release along with a Docker image for downstream projects like vllm-omni. In practice, however, there are gaps in this process. Taking v0.20.0 as an example, the timeline can be broken down into three phases:

1. Break Period

vLLM releases v0.20.0. vllm-omni adapts to v0.20.0 on day 0 or day 1, publishing v0.20.0rc1. Meanwhile, the main branch of vllm-ascend typically lags by 1–2 days, causing CI to break.

2. Downstream CI Installation Gap

Downstream CI installs vllm-ascend from main. Since the v0.20.0rc1 release of vllm-ascend is often delayed (usually arriving only 1–2 days before the next vLLM release), no vllm-ascend image that is compatible with vLLM v0.20.0 is available. Consequently, downstream projects must install vllm-ascend manually, facing three options:

  • Option 2.1: Continuously pull and build from vllm-ascend main. Build time is approximately 30 minutes.
  • Option 2.2: Use the vllm-ascend nightly Docker image (built from main). This requires pulling the latest image daily, resulting in a longer first-trigger PR time.
  • Option 2.3: vllm-ascend quickly publishes an rc0 image immediately after adopting v0.20.0 (ideal but not yet standard practice).

Current choice: We are using the vllm-ascend nightly image built from the main branch (Option 2.2).

3. Normal Operation

After vllm-ascend v0.20.0rc1 is officially released, downstream projects can use a fixed-version Docker image directly. Build time is reduced to approximately 10 minutes.

Contributor guide