[Feature] 多轮 context 历史消息剥图:只保留最后一条 user 的 image,更早的替换占位符
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 82/100
Research direction
Start in astrbot/core/provider/sources/openai_source.py at _query, immediately after _materialize_context_image_parts, and inspect how context_query becomes the LLM payload. Verify the history-filtering behavior for messages before the last user message, including empty content lists and text-only conversations. Done means historical image blocks become the specified placeholder while the last user message remains unchanged.
Written by the indexing model from the issue text.
Description
Description / 描述
当前问题:
astrbot/core/provider/sources/openai_source.py 在 _query 内,
_materialize_context_image_parts 后直接把整个 context_query 打包成 payloads
发给 LLM。所有历史消息(旧的 user / assistant)里的 image_url content
都会被重复发送。
多轮带图对话场景下,第 N 轮请求会包含全部 N-1 轮的历史图片 + 当前轮的图片,
vision tokens 累积爆炸。Anthropic Claude / OpenAI GPT-4 vision 单张图
~1500-3000 tokens(取决于分辨率),10 轮带图对话单次请求能轻松达到
数万 vision tokens。
建议:
在 _materialize_context_image_parts 后增加一步剥离:从 context_query
尾巴倒序找第一个 role='user' 的消息(保留它的图,因为是当前轮的视觉输入),
把它之前所有消息的 image_url 内容替换为 text 占位符 "[图片已省略]"。
逻辑示意:
_last_user_idx = -1
for _i in range(len(context_query) - 1, -1, -1):
if context_query[_i].get("role") == "user":
_last_user_idx = _i
break
if _last_user_idx > 0:
for _i in range(_last_user_idx):
_content = context_query[_i].get("content")
if isinstance(_content, list):
_filtered = [b for b in _content
if not (isinstance(b, dict)
and b.get("type") in ("image_url", "image"))]
if len(_filtered) != len(_content):
context_query[_i]["content"] = _filtered if _filtered else [
{"type": "text", "text": "[图片已省略]"}
]
影响范围:
- 只覆盖历史 image,当前轮(最后一条 user)的图不受影响 → vision 输入完整保留
- 历史 image 替换为占位符,LLM 仍知道"那一轮有图",不至于上下文断裂
- text-only 对话完全不受影响(context 里没 image_url 时 if 块不触发)
~10 行新增代码,无外部依赖。
Use Case / 使用场景
-
多轮图片细节讨论:
用户发图 → AI 评论 → 用户继续问"再仔细看"或"对比另一张" → AI 再评 → 反复
每轮都重新发全部历史图片,vision tokens 重复计费。 -
设计评审 / 形象评选 (MoSphere 实际场景):
用户陆续发多版设计稿 / logo 候选 / 形象 mockup,跟 AI 多轮讨论选哪个。
累积到第 5-10 轮时,单次请求 vision token 已超 30K。
剥图后每次请求只含当前轮的图(~3K vision tokens),节省 90%+。 -
截图聊天:
用户 share 截图 → AI 分析 → 后续对话不再涉及那张截图
历史截图本质是"已经被看过"的内容,没有重复发的价值
占位符 "[图片已省略]" 仍保留 LLM 知道"那条消息原本带图"的语义 -
主动消息 / cron 触发场景:
长期持续运行的 AstrBot,conversation 历史可能很长
即使每轮只一张图,30 天对话累积下来的图片量也很可观
剥图让 token 占用回到合理范围
实现简单(~10 行),保留语义完整性(占位符),LLM 体验不变。
愿意提 PR。
注:本优化跟 Anthropic prompt cache(另一个 issue)独立,组合使用收益叠加
(cache 减少 input cost,剥图减少 vision token)。
Willing to Submit PR? / 是否愿意提交PR?
- Yes, I am willing to submit a PR. / 是的,我愿意提交 PR。
Code of Conduct
- I have read and agree to abide by the project's Code of Conduct. /
- Dominant language
- Python
- Stars
- 40.7k
- Forks
- 2.9k
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 114
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from AstrBotDevs/AstrBot
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
AstrBotDevs/AstrBot#10033 · 3 comments ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
AstrBotDevs/AstrBot#9999 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
AstrBotDevs/AstrBot#9945 · 2 comments ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
AstrBotDevs/AstrBot#9938 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
AstrBotDevs/AstrBot#9929 · 2 comments ·
All issues in AstrBotDevs/AstrBot
Similar issues
-
documentation help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
simonw/sqlite-utils#872 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100