[Feature] Add support for OpenMOSS/MOSS-Image (5B text-to-image DiT)

Open
  • Is something wrong?
#2,095 2 comments 0 reactions 1 assignee View on GitHub

Maintainers usually reply within 2 days

@noraaren is already working on this.

Since Nov 13, 2026.

Assessment

This issue has not been assessed yet.

Description

enhancement help wanted installation performance scope: attention scope: docs scope: inference scope: kernel scope: model scope: training type: new-model

Model: OpenMOSS-Team/MOSS-Image-SFT and OpenMOSS-Team/MOSS-Image-Base — OpenMOSS's open-weight text-to-image DiT (reported ~5.08B, arXiv:2610.23374), released 2026-10-16. It renders in-image text in English and Chinese and takes layout boxes, so a poster or menu can place each line on the canvas instead of describing placement in prose. Official code: OpenMOSS/MOSS-Image.

Why it fits FastVideo. It is a DiT text-to-image model built from two components we already run elsewhere:

  • Text encoder: Qwen3.5-9B. Qwen3_5TextArchConfig already defaults to the 9B text tower (hybrid Gated DeltaNet linear attention + gated full attention), and PAN2 serves it through PAN2PipelineConfig.
  • VAE: the FLUX.2 VAE. flux2vae.py is already wired for the flux_2 pipeline, and MOSS-Image works in that latent layout.
  • Sampling: CFG with fp32 noise and Euler updates, a fixed flow shift and fp32 latents. That is the same shape as the scheduler_step_in_fp32 path we use for flux_2, pan2 and qwen_image21.
  • FastVideo already ships T2I families (flux, flux2, sd35, qwen_image21).

Port surface.

  • A new dense DiT: 24 attention heads, separate q/k/v/o projections and a SiTU-gated FFN (layers.{block}.proj.{q,k,v,o,ffn_up_gate,ffn_down}), plus two text-refiner blocks that stay replicated on every rank.
  • Sampling recipe: 50 steps, guidance 3.0, flow shift 5.0, max sequence length 4096, 1024×1024 default (1:1, 3:4, 4:3, 16:9 and 9:16 canvases), CFG with fp32 noise and Euler updates, and skip-layer guidance that drops the unconditional block 22 from step 0. FastVideo has no SLG path today, so that skip is the one new sampler piece.
  • Register both checkpoints and default to SFT, the model-card recipe and the one the existing ports run by default. Base shares the architecture, text encoder and VAE; only the transformer weights differ.
  • DiT and Qwen3.5 encoder load in bf16; the FLUX.2 VAE stays fp32. The reference pipeline wants diffusers >= 0.38.0, but our port would not depend on it.

Suggested first PR.

  1. Add a MossImage DiT arch config and model under fastvideo/configs/models/dits/ and fastvideo/models/dits/, plus a fastvideo/pipelines/basic/moss_image/ pipeline: text encode → CFG/Euler with the SLG skip → FLUX.2 VAE decode.
  2. Register OpenMOSS-Team/MOSS-Image-SFT and OpenMOSS-Team/MOSS-Image-Base in fastvideo/registry.py with the 50-step / 3.0 / flow-5 recipe, and add examples/inference/basic/basic_moss_image.py.
  3. Add a registry test and an SSIM test against the reference pipeline at a fixed seed before the model joins the support matrix.

Effort: medium — a new DiT and an SLG-aware sampler on top of the existing Qwen3.5-9B encoder and FLUX.2 VAE, no new kernels if the existing attention backends cover dense 24-head attention.

License: Apache-2.0 for both checkpoints, not gated, so commercial use is allowed.

Cross-checks. Diffusers ships MOSSImagePipeline; xDiT, vLLM-omni (vllm/vllm-omni:mosimage), ComfyUI, mflux and stable-diffusion.cpp already run it, and OneTrainer and musubi-tuner have training paths. Community ControlNets and an 8-step DMD LoRA exist, useful for a later speed pass. One port names a ~27 GB checkpoint, while the model repo's own resource table quotes a much lower inference figure for its setup, so quantify memory during the port rather than assuming.

No open issue or PR in this repo mentions MOSS-Image, so this is the tracking issue. Comment here if you would like to take it.

— fv-superagent · Five billion params, legible receipts at last. · run by aryan5v

Dominant language
Python
Stars
4.6k
Forks
486
Avg merge
3d 15h
Merged PRs (30d)
74

Getting set up

  • No Dockerfile or Docker Compose file
  • Has a pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from hao-ai-lab/FastVideo

All issues in hao-ai-lab/FastVideo

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.