Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

[BUG] rag: recursive chunk expansion returns duplicate IDs

Open Beginner friendly
  • Is something wrong?
#1,465 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@LJK-ikun is already working on this.

Since Oct 21, 2026.

  • #1466 by @LJK-ikun — open

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
80/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
node.js, typescript
Domain
data

Research direction

Start by locating RecursiveChunker.chunk() in the @voltagent/rag source and trace how sentence-group expansion and token fallback assign IDs. Add public API regressions for sentence-only and mixed sentence/token output, as described in the issue. Done when IDs are unique within one call while chunk contents and overlap behavior remain unchanged.

Written by the indexing model from the issue text.

Description

bug
Describe the bug

RecursiveChunker.chunk() returns duplicate chunk IDs when a long paragraph is split into several sentence groups. Three different returned chunks can all have id: "recursive-0". An ingestion application that keys records by the returned ID cannot distinguish those chunks; this report does not claim to have tested a particular vector database.

Steps To Reproduce

Using the unchanged public exports of @voltagent/rag on upstream main at 72a46c76b4507e1f98b41d5050a92bd30587eb70:

import { RecursiveChunker, whitespaceTokenizer } from "@voltagent/rag";

const chunks = new RecursiveChunker(whitespaceTokenizer).chunk(
  "Alpha one. Beta two. Gamma three. Delta four.",
  { maxTokens: 4 },
);
console.log(chunks.map(({ id, content }) => ({ id, content })));
console.log(new Set(chunks.map((chunk) => chunk.id)).size, chunks.length);

Actual output:

recursive-0: Alpha one. Beta two.
recursive-0: Beta two. Gamma three.
recursive-0: Gamma three. Delta four.
1 3
Expected behavior

IDs should distinguish each chunk returned by one call, including sentence groups and token fallback chunks across paragraphs. Chunk contents and overlap behavior should stay the same.

Packages

@voltagent/rag 1.0.2 source on current main; Node.js 24.19.0, Windows. Reproduced with the actual package source and its declared js-tiktoken dependency, not a recreated implementation. No model, API key, or external service is involved.

Additional Context

The sentence-group expansion computes IDs from chunks.length before any expanded chunks are appended. I plan to assign final IDs after expansion and add public-API regressions for sentence-only and mixed sentence/token output. Searched existing issues and open PRs for RecursiveChunker and chunk IDs; did not find a matching fix. PR #893 introduces a separate document utility package, while #1459 and #1463 address JSON handling.

AI disclosure: Codex investigated and reproduced this under LJK-ikun's direction. This report describes the observed local output; it does not assert independent human testing.

Dominant language
TypeScript
Stars
10.9k
Forks
1.3k
Avg merge
4h 57m
Merged PRs (30d)
2

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from VoltAgent/voltagent

All issues in VoltAgent/voltagent

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.