[RFC]: SenseNova-U1Optimization Roadmap

Motivation.

Related: https://github.com/vllm-project/vllm-omni/pull/3319

Inference Performance

• Online dynamic batching — batch multiple requests with heterogeneous prompt lengths and image resolutions into a single forward pass; handle variable-length KV prefix packing and mixed per-request CFG scales https://github.com/vllm-project/vllm-omni/pull/4156
• MoT kernel fusion — fuse the dual QKV/MLP branch dispatch (und/gen) into a single gather-scatter GEMM; fuse Q/K RMS norms and 3D RoPE (t, h, w) into single custom kernels @lsyyysky
• Cache-DiT acceleration — register SenseNova-U1 in CUSTOM_DIT_ENABLERS and implement a SensenovaCachedAdapter; the KV-prefix architecture requires caching the image-token hidden residuals, not the full KV cache @fywc
• TeaCache acceleration — add SenseNova-U1 feature extractors and coefficient estimators to the TeaCache registry; calibrate rel_l1_thresh per resolution bucket #4164

---

Parallelism

• Sequence Parallelism (SP / USP) — define _sp_plan on SenseNovaU1Model to shard image-token hidden states ([B, L_image, C]) across Ulysses or ring-attention ranks; replicated prefix KV cache and text tokens must remain unsharded @knitcapcat-amd
• CFG-Parallel — apply CFGParallelMixin to SenseNovaU1Pipeline; route cond/uncond (and img_cond for IT2I) forwards across cfg_parallel_size ranks instead of sequential execution @sphinxkkkbc https://github.com/vllm-project/vllm-omni/pull/4188

---

Deployment Topology

• AR+DiT stage separation — split the prefix-encode + think-mode AR phase (AR stage) from the denoising loop (DiT stage) into separate processes, following the Bagel stage model; wire stage_diffusion_proc.py and add a SensenovaU1StageInputProcessor @fywc
• Prefix KV cache across requests — within the AR stage, cache prefix KV tensors keyed by (prompt hash, image hash) so identical prompts skip re-encoding across requests
• xARyDiT deployment — support deploying x AR replicas + y DiT replicas with an async queue between stages; AR replicas can serve think-token generation while DiT replicas run denoising

---

Quantization

• FP8 offline quantization — add SenseNova-U1 to the ModelOpt/quantization workflow; handle MoT dual-branch weight tensors (both branches must be quantized together, scale tensors per-branch)
• FP8 online (dynamic) quantization — enable per-token dynamic FP8 activation quantization for the gen-path QKV and MLP GEMMs during the denoising loop https://github.com/vllm-project/vllm-omni/pull/3943
• INT8 / GGUF support — extend DiffusionInt8Config and GGUF adapters to cover SenseNova-U1 weight layout (stacked QKV, MoT dual branches, gate_up_proj)

---

Adapters

• LoRA support — wire DiffusionLoRAManager to SenseNovaU1ForCausalLM; add LoRA adapters for both MoT branches (qkv_proj, qkv_proj_mot_gen, gate_up_proj, gate_up_proj_mot_gen, o_proj, down_proj); support dynamic adapter swapping at request time https://github.com/vllm-project/vllm-omni/pull/3971

---

Training Integration

• verl-omni integration — expose SenseNova-U1 as a rollout model in verl's engine interface; implement @steven-kid generate_sequences using the vLLM-Omni async engine with the denoising loop mapped to verl-omni's step interface
• Flow-GRPO — implement flow-matching policy gradient training: use denoising trajectory log-probabilities under the flow ODE as the policy, with a KL penalty against a reference model snapshot
• Dance-GRPO — support online reward-weighted denoising trajectory sampling; define reward functions for image quality and prompt alignment over the denoising distribution
• Mix-GRPO — support joint AR (think-token) and DiT (denoising) policy optimization in a single GRPO pass; define mixed reward signals combining text-generation and image-quality terms

Proposed Change.

Please provide the detailed design document of the RFC using the template.

Feedback Period.

No response

CC List.

No response

Any Other Things.

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.