[New Model]: SongGen — Single-Stage Auto-Regressive Transformer for Text-to-Song

The model to consider.

SongGen (ICML 2025) by Zihan Liu et al. is a single-stage auto-regressive transformer for text-to-song generation with lyrics + text description + optional reference voice control.

Model	Weights	Params	Mode
SongGen Mixed Pro	Public (Apache-2.0)	1.3B	Single-track (mixed vocals + accompaniment)
SongGen Interleaving A-V	Public (Apache-2.0)	1.3B	Dual-track (separate vocal / accompaniment)

The closest model vllm-omni already supports.

Qwen3-TTS / Qwen3-Omni Talker — both are decoder-only AR transformers that emit audio codec tokens decoded by a downstream codec. SongGen follows a very similar paradigm:

1. Single-stage AR: text + lyrics conditioning, AR generation of audio codec tokens, no separate diffusion head.
2. Codec backbone: uses X-Codec (xcodec_hubert_general_audio_v2) for audio token <-> waveform.
3. Built on Parler-TTS: which already follows the Hugging Face ForConditionalGeneration AR pattern, close to existing TTS model wrappers in vllm-omni.

What's your difficulty of supporting the model you want?

Architecture overview

• Decoder-only AR transformer conditioned on (a) lyrics tokens (VoiceBpeTokenizer), (b) free-form music description text, (c) optional reference voice features.
• Output: discrete X-Codec tokens, decoded to 24kHz waveform by the X-Codec decoder.
• Two model heads:
  • SongGenMixedForConditionalGeneration (single audio stream)
  • SongGenDualTrackForConditionalGeneration (interleaved A/V tokens, returns vocal and accompaniment arrays separately)

Integration notes

• Inference path is model.generate(...) with sdpa attention, mirroring how Qwen3-TTS Talker is currently invoked. Should map naturally to a vllm-omni AR stage that emits codec tokens, plus a Code2Wav-style decode stage wrapping X-Codec.
• Reference-voice conditioning runs an upstream vocal separation step in the official processor (separate=True). For serving we likely want this preprocessed offline.
• Training code is open-sourced (released 2025/7/4), useful for any vllm-omni-side tuning later.

Known limitations from the authors

• English lyrics only.
• Maximum song duration ~30s.
• Trained on ~2k hours; authors explicitly invite scaling collaborations.

Use case and motivation

1. Cleanest AR fit for vllm-omni: among current open-source song-generation models, SongGen is the most "decoder-only LM emitting codec tokens" in spirit — the same shape as our Qwen3-TTS / Qwen3-Omni serving path.
2. Apache-2.0 license: weights, code, and Mixed Pro / Interleaving A-V checkpoints all public.
3. Dual-track output is a capability no current vllm-omni TTS model exposes, useful for downstream music applications.
4. Manageable scope: 1.3B params, 30s clips — a tractable starting point to validate vllm-omni's song-generation pipeline before attempting larger systems.

References

• Paper: https://arxiv.org/abs/2502.13128
• GitHub: https://github.com/LiuZH-19/SongGen
• Demo: https://liuzh-19.github.io/SongGen/
• HF collection: https://huggingface.co/collections/LiuZH-19/songgen-a-single-stage-auto-regressive-transformer-for-text-6867ec21169d808034f6d252
• X-Codec: https://github.com/zhenye234/xcodec

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.