Support Perceptual Flow Matching (PFM) for Few-Step Generation and Distillation
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 48/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- python
- Domain
- machine-learning
Research direction
No files or tests are named. Start by locating the current SFT training pipeline and its training-loss implementation, then compare it with the authors' existing PFM implementation. Done means PFM can replace the standard loss in the pipeline and support the validated SD3, Qwen-Image-Edit, and Wan use cases.
Written by the indexing model from the issue text.
Description
Hi! We recently published our work, Perceptual Flow Matching for Few-Step Generative Modeling: https://arxiv.org/abs/2607.03524.
In this work, we show that simply replacing the standard MSE loss with a perceptual loss during flow matching training can produce a strong few-step generator with only a few hundred training steps. We have validated the method on SD3, Qwen-Image-Edit, and Wan. The Qwen-Image-Edit and Wan experiments were both implemented based on DiffSynth-Studio.
I was wondering whether you would be interested in integrating PFM into DiffSynth-Studio. The method is highly compatible with the current SFT training pipeline, as it only requires replacing the training loss while leaving the rest of the framework unchanged.
If this sounds interesting, I'd be happy to help integrate our existing implementation and submit a PR for the project.
- Dominant language
- Python
- Stars
- 13.2k
- Forks
- 1.3k
- Avg merge
- 22h 15m
- Merged PRs (30d)
- 31
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from modelscope/DiffSynth-Studio
-
WanToDance: hardcoded `device='cuda'` in music encoder construction crashes model loading on Ascend NPUPossibly taken @li-lizhe claimed this 13 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
modelscope/DiffSynth-Studio#1707 · 1 comment ·
Maintainers usually reply within 1 day
-
Partial download surfaces as "Cannot detect the model type" rather than a download errorPossibly taken @MohammadHijjawi97 claimed this 11 days ago. Open
Difficulty 1/5 1-3 hours Newbie friendliness 78/100
modelscope/DiffSynth-Studio#1668 ·
Maintainers usually reply within 1 day
-
bfloat16训练时段错误: 数组越界的一种方案May be free again A pull request for this issue was closed without being merged. Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
modelscope/DiffSynth-Studio#1499 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 78/100
modelscope/DiffSynth-Studio#1373 · 5 comments · 1 reaction ·
Maintainers usually reply within 1 day
-
[Reproducibility] Training script / config for MiniMax-H3-TrainingAdapter (DeCFG adapter) itselfOpen
Difficulty 5/5 Over a week Newbie friendliness 35/100
modelscope/DiffSynth-Studio#1713 · 1 comment ·
Maintainers usually reply within 1 day
All issues in modelscope/DiffSynth-Studio
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Maintainers usually reply within 3 days
-
Negation with "not" and "no" is ignored during sentiment analysisPossibly taken @vivek-3728 claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
techcsispit/mess-mood#11 · 1 comment ·
-
changelog investigate
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
ramnes/notion-sdk-py#408 ·
-
good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 83/100
btclib-org/btclib-wallet#267 ·
Maintainers usually reply within 1 day
-
good first issue tech-debt
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
knnmelprop/YAADO#111 ·