[Looking for community contribution] support Wan 2.2 S2V: an audio-driven cinematic video generation model · huggingface/diffusers#12257

(4 评论) (1 反应) (0 负责人)Python (22,190 star) (4,562 fork)batch import

Good second issuecontributions-welcomehelp wanted

描述

We're super excited about the Wan 2.2 S2V (Speech-to-Video) model and want to get it integrated into Diffusers! This would be an amazing addition, and we're looking for experienced community contributors to help make this happen.

Project Page: https://humanaigc.github.io/wan-s2v-webpage/
Source Code: https://github.com/Wan-Video/Wan2.2#run-speech-to-video-generation
Model Weights: https://huggingface.co/Wan-AI/Wan2.2-S2V-14B

This is a priority for us, so we will try review fast and actively collabrate with you throughout the process :)

贡献者指南

技术栈: pythonpytorch
领域: machine learningai
议题类型: feature
难度: 4
预计时间: over 1 week
活动状态: fresh
清晰度: mostly clear
前置要求: Familiarity with diffusers libraryExperience with PyTorchUnderstanding of diffusion modelsAudio processing basics
新手友好度: 15
研究方向: The issue requests integration of the Wan2.2 S2V model into diffusers. Start by studying the project page and source code links provided. Examine existing model pipelines in diffusers (e.g., text to video or image to video) to understand the expected pattern. Review the model's architecture and weight loading. Then propose a pipeline that handles audio input and generates video. Consult the diffusers contribution guide for new models. The maintainers have promised fast review; engage with them for design decisions.