Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

[Bug] 知识库 URL 清洗回退时丢失 chunk_size/chunk_overlap，按默认参数入库

Open Beginner friendly
  • Is something wrong?
#9,999 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
86/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
backend

Research direction

Start at astrbot/core/knowledge_base/kb_helper.py lines 835 and 892, then inspect the shared chunker setup in kb_helper.py:275 and kb_mgr.py:22. Run the supplied standard-library reproduction and add coverage for missing-provider and provider-error fallbacks; done means both preserve the requested 128/16 chunking, matching the non-cleaning path.

Written by the indexing model from the issue text.

Description

bug
问题描述

_clean_and_rechunk_content() 在未配置清洗 provider 或获取 provider 失败时，执行 self.chunker.chunk(content)，丢失本次传入的分块参数。未启用清洗的正常路径则会正确传递参数。

知识库管理器的共享分块器默认为 500/100。单元复现请求 128/16，两个回退分支实际均输出最大长度 500 的块。后续 upload_document(pre_chunked_text=...) 直接使用这些块，因此后续传入的大小参数不会补救这一偏差。

代码定位
如何复现？
  1. 对 1000 字符无分隔文本调用清洗流程，设置 chunk_size=128, chunk_overlap=16。
  2. 启用清洗但不传 provider ID，观察回退块长度。
  3. 对照指定 provider ID，但令 get_provider_by_id 抛出异常。

预期：回退继续保留 128/16 的本次分块参数。实际：两条路径均恢复到分块器默认 500/100。

下面脚本已在该提交的仓库根目录用 Python 3.12.14 独立运行。为避免加载完整 AstrBot 和外部依赖，通过 AST 读取并执行仓库原始类定义；方法体保持不变，日志、数据库或上游模型按用例使用 mock。断言用于确认当前缺陷与对照结果，无真实模型调用或线上平台测试。

可运行的源码级复现（仅 Python 标准库）

保存为临时 Python 脚本，在 AstrBot 仓库根目录运行：

import ast, asyncio, abc, copy, functools, re, time, dataclasses
from pathlib import Path
from types import SimpleNamespace as NS
from unittest.mock import AsyncMock, Mock
ROOT = Path.cwd()
def load(path, names, env=None):
    tree = ast.parse((ROOT/path).read_text())
    selected = [n for n in tree.body if isinstance(n, (ast.ClassDef, ast.FunctionDef, ast.AsyncFunctionDef)) and n.name in names]
    scope = dict(__name__='__main__', asyncio=asyncio, abc=abc, copy=copy, functools=functools, re=re, time=time, dataclass=dataclasses.dataclass, logger=Mock(), BaseChunker=object, deprecated=lambda **kw: lambda f:f)
    scope.update(env or {})
    mod=ast.Module(body=[ast.ImportFrom(module='__future__', names=[ast.alias(name='annotations')],level=0), *selected],type_ignores=[])
    exec(compile(ast.fix_missing_locations(mod),str(ROOT/path),'exec'),scope)
    return scope
async def main():
    r=load('astrbot/core/knowledge_base/chunking/recursive.py',['RecursiveCharacterChunker'])['RecursiveCharacterChunker']
    s=load('astrbot/core/knowledge_base/kb_helper.py',['KBHelper'])
    helper=object.__new__(s['KBHelper']); helper.chunker=r()
    chunks=await helper._clean_and_rechunk_content('X'*1000,'https://example.invalid',enable_cleaning=True,cleaning_provider_id=None,chunk_size=128,chunk_overlap=16)
    print('5 cleaning fallback: requested size=128, actual max=',max(map(len,chunks)))
    assert max(map(len,chunks))==500
    helper.prov_mgr=NS(get_provider_by_id=AsyncMock(side_effect=RuntimeError('provider unavailable')))
    chunks=await helper._clean_and_rechunk_content('X'*1000,'https://example.invalid',enable_cleaning=True,cleaning_provider_id='broken',chunk_size=128,chunk_overlap=16)
    assert max(map(len,chunks))==500
asyncio.run(main())
AstrBot 版本

4.28.0，master 提交 6494ac22e536d38ef6e07014d27b808d8c114bd2（2026-09-09 扫描）。

操作系统

Linux，Python 3.12.14。

部署方式

源码级单元复现；未启动完整服务。

使用的消息平台适配器

无；核心代码测试，不加载插件或平台适配器。

错误日志

以下为实际运行脚本的输出，非生产日志：

5 cleaning fallback: requested size=128, actual max= 500
建议修复与验证

两处回退调用均传入 chunk_size=chunk_size, chunk_overlap=chunk_overlap。补充缺少 provider 和 provider 获取异常两个分支的参数保留测试，并与未启用清洗的结果比较。

重复检查

已检索清洗/分块、相关函数及开放 PR，未发现相同回退参数丢失问题。该复现采用纯字符分割，不依赖 #9901 的 overlap 合并缺陷。

检查清单
  • 已搜索相关 issue 和开放 PR，未找到相同根因的报告。
  • 复现未加载插件，问题位于 AstrBot 核心代码。
  • 已阅读项目贡献指南及行为准则。
  • 已明确区分源码级复现、mock 边界与未验证的线上场景。
Dominant language
Python
Stars
41.1k
Forks
3k
Avg merge
1d 2h
Merged PRs (30d)
121

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from AstrBotDevs/AstrBot

All issues in AstrBotDevs/AstrBot

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.