Ref2VA: multi-subject voice identity can bleed between reference-image subjects
还没有人认领这个 Issue。
评估
- 难度
- 5/5
- 预计耗时
- 一周以上
- 新手友好度
- 35/100
- Issue 类型
- 缺陷
- 描述清晰度
- 需要澄清
- 活跃度
- 活跃
- 技术栈
- python
- 领域
- ai, audio-video-rtc
调研方向
报告中没有指定文件、测试或代码入口点。首先使用交替发言者和显式语音条件控制来复现 Ref2VA 和 FL2VA 场景；完成的标准是每个主体始终保留其分配到的声音，且不会泄漏到另一位发言者。
由索引模型根据 Issue 内容生成。
描述
I’m seeing a repeatable speaker-routing issue with MiniMax H3 Ref2VA in multi-character dialogue scenes.
Visual subject binding works extremely well: multiple characters referenced from separate images remain visually distinct and consistent. However, the audio branch can fail to keep different voice identities attached to the correct subjects. One subject’s voice characteristics can bleed into another subject, or one voice can effectively dominate both speakers.
Example setup:
<Subject 1> = female character from <Picture 1>
Voice A = distinct British female voice anchor
<Subject 2> = female character from <Picture 2>
Voice B = distinct American female voice anchor
Dialogue alternates: Subject 2 → Subject 1 → Subject 2.
The visual identities and speaker/lip assignments remain correct, but Voice B can bleed into Subject 1, causing both subjects to share or partially share the same vocal identity, or Subject 1 uses a completely different voice that wasn't prompted.
Update: I originally reported this as a Ref2VA-specific issue, but I have now reproduced the same multi-speaker voice bleed using FL2VA from a single composed first frame with no last frame.
In FL2VA, explicitly conditioning Subject 2 to speak in an American voice causes Subject 1’s voice to become American as well. Conversely, explicitly conditioning Subject 1 to speak in an Elizabeth Swann / Keira Knightley-style British voice restores Subject 1’s voice, but Subject 2 also shifts to a British voice.
The issue also persists with only one speaker transition (A → B), with explicit tags, and with pure <Subject 1> / <Subject 2> speaker labels.
This suggests the issue may not be specific to Ref2VA reference routing. It may instead involve multi-speaker voice conditioning being applied too globally across the audio stream rather than remaining reliably bound to an individual speaker.
For comparison, other two-character combinations can work correctly, so this does not appear to be a universal inability to generate two distinct voices.
- 主要语言
- Python
- 星标
- 10.6k
- 派生
- 807
- PR 合并指标
- 30 天内没有已合并 PR
环境准备
这个项目没有提供开发容器、Dockerfile 或贡献指南，环境需要你自己搭建：先看它的 README，通用步骤见我们的新手贡献指南。
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
MiniMax-AI/MiniMax-H3 的其他 Issue
-
难度 4/5 3-5 天 新手友好度 45/100
MiniMax-AI/MiniMax-H3#96 ·
-
难度 4/5 3-5 天 新手友好度 35/100
MiniMax-AI/MiniMax-H3#95 ·
-
难度 5/5 一周以上 新手友好度 1/100
MiniMax-AI/MiniMax-H3#94 ·
-
loss of init.py未关闭
难度 3/5 半天 新手友好度 15/100
MiniMax-AI/MiniMax-H3#92 · 1 条评论 ·
-
难度 5/5 一周以上 新手友好度 5/100
MiniMax-AI/MiniMax-H3#89 ·
查看 MiniMax-AI/MiniMax-H3 的全部 Issue
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 68/100
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 76/100
scikit-learn-contrib/category_encoders#553 ·
维护者通常 1 天内回复
-
flaky-watch
难度 2/5 1-3 小时 新手友好度 88/100
btclib-org/btclib#2839 ·
维护者通常 1 天内回复
-
BarrelList constructor drops false-valued iterable contents可能已有人在做 @Gambit-Checkmate 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 76/100
维护者通常 19 天内回复
-
难度 2/5 1-3 小时 新手友好度 77/100
vllm-project/vllm#63863 ·
维护者通常 1 天内回复