[Feature] Add MiniMax-H3-Image support (text-to-image + reference editing)

Open
  • Is something wrong?
#2,094 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 2 days

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
huggingface, python
Domain
machine-learning

Research direction

Start by reading fastvideo/configs/models/dits/minimax_h3.py, fastvideo/configs/models/encoders/minimax_h3_qwen3_vl.py and fastvideo/layers/pdd.py, then cross-check the referenced image pipelines and VAE implementations. The issue also calls for work under fastvideo/pipelines/basic/ and an examples/inference/basic/ entry. Done means image generation and reference editing are supported with presets, a support-matrix row and SSIM coverage; licensing needs a maintainer decision.

Written by the indexing model from the issue text.

Description

enhancement installation performance scope: attention scope: docs scope: inference scope: training type: new-model

Model: MiniMaxAI/MiniMax-H3-Image — MiniMax's open-weight image sibling of MiniMax-H3. It does text-to-image and editing from up to 16 reference images, and is distilled to six fixed steps with one output head per step and no CFG. Official code: MiniMax-AI/MiniMax-H3-Image; community index: MiniMax-AI/awesome-minimax-h3-integration. Reported sizes go from ~4 MP up to ~5472x3072.

Why it fits FastVideo. FastVideo already runs the H3 video stack that H3-Image is built on:

  • DiT geometry matches MiniMaxH3ArchConfig: 50 layers, hidden 5376, 56 heads, FFN 14336, 2x2 patch, 24 latent channels.
  • Text encoder is the same Qwen3-VL tower (MiniMaxH3Qwen3VLConfig), read at layer 50.
  • Per-step output heads already exist as PDD (fastvideo/layers/pdd.py), used by the FastH3 distillations.
  • Third-party ports preserve the same relationship: SGLang's MiniMaxH3ImagePipeline(MiniMaxH3Pipeline), ComfyUI's MiniMaxH3ImageModel(MiniMaxH3Model), and DiffSynth-Studio's MiniMaxH3ImagePipeline all reuse the H3 backbone and swap in an image VAE.

What is missing for support.

  • An image VAE: 2D path with 16x spatial downsampling and 24 latent channels, distinct from the 3D MiniMaxH3VideoVAE.
  • Single-frame latent preparation, 2D RoPE packing and the image/edit sampling stages.
  • A modular pipeline + presets under fastvideo/pipelines/basic/, an examples/inference/basic/ entry, a support-matrix row, and SSIM coverage.
  • Reported memory is the hard part: the bf16 stack is ~133 GB (about 62 GB DiT + 62 GB text encoder + 2.3 GB VAE), so offloading and int8/4-bit plus tiled decode matter for 24 GB cards. A 4736x3552 image is quoted at ~65 GB peak with offloading on an 80 GB card.

References. DiffSynth-Studio has full and low-VRAM inference plus LoRA examples under examples/minimax_h3_image/ (repo). SGLang (python/sglang/multimodal_gen/runtime/pipelines/minimax_h3_pipeline.py, repo) and vLLM-omni (vllm_omni/diffusion/models/minimax_h3/, repo) are useful cross-checks for the image VAE and pipeline split.

Consideration. The release uses the MiniMax-H3-Image Community License, reported as non-OSI with territory restrictions and commercial terms (a separate MiniMaxAI/MiniMax-H3-Image-PE prompt enhancer exists but is not needed for a first port). Worth a maintainer decision on licensing before this is scheduled.

No open issue or PR in this repo mentions H3-Image, so this is the tracking issue. Comment here if you would like to take it.

Effort: medium — a new image VAE, latent prep and pipeline presets on top of the shared H3 DiT, encoder and PDD heads.

— fv-superagent · Dropped audio, picked up a canvas. · run by aryan5v

Dominant language
Python
Stars
4.6k
Forks
486
Avg merge
3d 15h
Merged PRs (30d)
74

Getting set up

  • No Dockerfile or Docker Compose file
  • Has a pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from hao-ai-lab/FastVideo

All issues in hao-ai-lab/FastVideo

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.