text chat: expose image input (enable multimodal M3, incl. multi-image)
还没有人认领这个 Issue。
评估
- 难度
- 3/5
- 预计耗时
- 1-2 天
- 新手友好度
- 74/100
- Issue 类型
- 功能
- 描述清晰度
- 描述清楚
- 活跃度
- 冷清
- 技术栈
- typescript
调研方向
从 src/commands/text/chat.ts 开始,重点查看 parseMessages 和 chat body builder,然后检查 src/utils/image.ts 以及现有的 vision describe image 处理。添加可重复指定的本地路径或 URL 图片输入,使其生成 Anthropic image blocks,支持多张图片,并在需要时默认使用 MiniMax-M3;完成标准是文本聊天无需 messages JSON 文件即可发送一张或多张图片。
由索引模型根据 Issue 内容生成。
描述
Summary
mmx text chat uses the multimodal MiniMax-M3 model by default, but the CLI exposes no image input flag. Users cannot send an image (let alone multiple images) to M3 through text chat without hand-writing a base64 messages JSON file. Since M3 is multimodal, this is a significant hidden capability.
Current behavior
mmx text chatonly accepts text:--message,--messages-file,--system. There is no--imageflag.parseMessages(src/commands/text/chat.ts) passescontentthrough asstring | ContentBlock[], so image blocks can reach the API via--messages-file— but the CLI does no image handling (no path→base64 conversion, unlikevision describe).- For single-image description there is
mmx vision describe(which hits/v1/coding_plan/vlm, single-image only). - No CLI path exists for multi-image input (compare/diff/joint analysis of 2+ images in one call), even though M3 supports it.
Expected behavior
A first-class image input on text chat, e.g.:
# single image
mmx text chat --model MiniMax-M3 --image ./photo.jpg --message "What breed is this dog?"
# multiple images (repeatable)
mmx text chat --model MiniMax-M3 \
--image ./before.png --image ./after.png \
--message "List every visual difference between these two."
The flag should accept local paths / http(s) URLs and auto base64-encode them (reusing toDataUri from src/utils/image.ts), then inject them as image content blocks alongside the text message.
Evidence — multi-image already works via M3
I verified that M3 accepts multiple images in one call through mmx text chat --messages-file. Example (CN region, API key auth):
node -e '
const fs = require("fs");
const img = (p) => ({ type: "image", source: { type: "base64", media_type: "image/png", data: fs.readFileSync(p).toString("base64") } });
fs.writeFileSync("/tmp/m.json", JSON.stringify([{
role: "user",
content: [
{ type: "text", text: "I am giving you TWO images. Describe one detail unique to each." },
img("/tmp/a.png"), img("/tmp/b.png")
]
}]));
'
mmx text chat --model MiniMax-M3 --messages-file /tmp/m.json --non-interactive --quiet
# → M3 correctly describes both images and distinguishes them
Format gotcha worth surfacing
mmx text chat posts to the Anthropic /messages endpoint (chatEndpoint returns ${baseUrl}/anthropic/v1/messages), not the OpenAI /chat/completions endpoint. So image content blocks must use the Anthropic shape:
// ❌ OpenAI shape — rejected: "unsupported content type 'image_url'"
{ "type": "image_url", "image_url": { "url": "data:..." } }
// ✅ Anthropic shape — works
{ "type": "image", "source": { "type": "base64", "media_type": "image/png", "data": "<base64>" } }
The OpenAI-compatible image format documented at https://platform.minimaxi.com/docs/api-reference/text-openai-api (type: "image_url") does not work through this CLI's text chat, because the CLI routes to the Anthropic endpoint. This mismatch took a while to debug — a --image flag that auto-formats correctly (or at least a doc note) would help a lot.
Suggested implementation
- Add a repeatable
--image <path-or-url>flag totext chat. - In
parseMessages/ the chat body builder, convert each--imageviatoDataUri, then append{ type: "image", source: { type: "base64", media_type, data } }blocks to the user message'scontent(convertingcontentfrom string to array when images are present). - When
--imageis present, default--modeltoMiniMax-M3if not set. - Optionally reuse the same
--imageflag on a futurevisionsubcommand for multi-image, since M3's chat path strictly supersedes the single-image/vlmendpoint for multi-image use cases.
Happy to open a PR if this design sounds reasonable.
- 主要语言
- TypeScript
- 星标
- 2.2k
- 派生
- 181
- 平均合并
- 9 小时 4 分钟
- 30 天内合并 PR
- 13
贡献指南
这个仓库没有索引到贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
MiniMax-AI/cli 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 84/100
MiniMax-AI/cli#259 · 1 条评论 ·
-
难度 2/5 1-3 小时 新手友好度 88/100
MiniMax-AI/cli#258 ·
-
难度 2/5 1-3 小时 新手友好度 35/100
MiniMax-AI/cli#265 ·
-
难度 5/5 一周以上 新手友好度 25/100
MiniMax-AI/cli#260 ·
-
难度 3/5 1-2 天 新手友好度 76/100
MiniMax-AI/cli#257 · 1 条评论 ·
相似的 Issue
-
VerificationGate: ATTRIBUTION quote guard never matches a normal quotation (\b around the quote) 未关闭
难度 2/5 1-3 小时 新手友好度 75/100
danielmiessler/LifeOS#2234 ·
-
T: Bug
难度 2/5 1-3 小时 新手友好度 75/100
-
难度 2/5 1-3 小时 新手友好度 65/100
-
难度 1/5 1 小时以内 新手友好度 85/100
-
Mend: dependency security vulnerability untriaged
难度 2/5 1-3 小时 新手友好度 70/100