orama-db: `orama-db.json` changes on every build
维护者通常 3 天内回复
评估
- 难度
- 2/5
- 预计耗时
- 1-3 小时
- 新手友好度
- 35/100
- Issue 类型
- 缺陷
- 描述清晰度
- 描述清楚
- 活跃度
- 停滞
- 技术栈
- javascript, node.js
调研方向
Start with packages/react/src/orama-db/generate.mjs: read how it builds documents, calls insertMultiple, and saves the index; the issue also points to useOrama.mjs to check how search uses results. Add stable ids and sort the flattened documents by href, then create the suggested generator tests for repeatability, input order, and search behavior. Run node --run test and confirm the generated output is stable; pull request #1190 is already open against this issue.
由索引模型根据 Issue 内容生成。
描述
Running a build twice on the same docs gives two different orama-db.json files, for two reasons:
- Timestamp-based ids. The
orama-dbgenerator inserts documents without anid, so Orama 3.x makes one up from the current time and a counter (e.g.20003290-1). That id is saved in the index (internalDocumentIDStore.internalIdToId), so the file changes on every build. - Insertion order. Orama's saved index follows the order documents were inserted, and the generator's input order can change between builds (see #1183).
The search index is several MB, so a file that changes on every deploy wastes caching, and CI can't compare it byte for byte.
Where to look
packages/react/src/orama-db/generate.mjs:27-44: builds the documents (title,description,href,siteSection) with noid, in input order.- The same file, line 40:
href(${entry.path}.html#${entry.heading.data.slug}), which makes a natural stable id. - The same file, line 47:
insertMultiple(db, documents); lines 49-56:save(db)is written toorama-db.json. packages/react/src/html/ui/hooks/useOrama.mjs:26-41: the website loads the saved index and searches it. It doesn't depend on the id format, and results are rendered fromhit.document.
How to reproduce
From packages/react (where @orama/orama resolves), save this as repro.mjs and run node repro.mjs a few times:
import { createHash } from 'node:crypto';
import { create, insertMultiple, save } from '@orama/orama';
const docs = [
{ title: 'fs', description: 'File system', href: 'fs.html#fs', siteSection: 'fs' },
{ title: 'fs > readFile', description: 'Reads a file', href: 'fs.html#readfile', siteSection: 'fs' },
];
const db = create({
schema: { title: 'string', description: 'string', href: 'string', siteSection: 'string' },
});
await insertMultiple(db, docs); // try: docs.map(doc => ({ id: doc.href, ...doc }))
const out = JSON.stringify(save(db));
console.log(createHash('sha256').update(out).digest('hex').slice(0, 16));
Each run prints a different hash. With the commented id: doc.href line, every run prints the same one, until you reverse docs, which changes it again.
Suggested approach
Only packages/react/src/orama-db/generate.mjs needs to change:
- Add
id: hrefto each document (Orama requires string ids, andhrefis one). - Sort the flattened documents by
hrefbefore inserting them, comparing with<and>(orlocaleCompare(..., 'en')) so the order doesn't depend on the machine's locale.
Keep the existing flatMap as it is, and sort its result, not the input: buildHierarchicalTitle walks each module's headings in document order to build titles like "Class > Method".
How to test
There's no generator test for orama-db yet. Create packages/react/src/orama-db/__tests__/generate.test.mjs, setting up the config like llms-txt-full/__tests__/generate.test.mjs does (setConfig({ target: ['orama-db'] }), then set output to undefined so generate() just returns the saved index). With a few fake metadata entries from two modules, test that:
- Two runs give the same output: compare the
JSON.stringifyof twogenerate()results. This already fails today within a single process. - Input order doesn't matter: put one module's entries before the other's, keeping each module's own entries in order, and compare.
- Search still works:
load()the result intocreate({ schema: {} }),search()for a title word, and checkhits[0].document.href.
Run everything with node --run test.
Things to watch out for
- Ids must be unique, or Orama throws
DOCUMENT_ALREADY_EXISTSand the build fails. Slugs are deduplicated per file, so hrefs should be unique. If the full Node.js docs build throws anyway, point out the duplicate in your PR rather than going back to automatic ids. save()doesn't include the database's own id, so that one doesn't matter.- The output is only identical for the same
@orama/oramaversion; a dependency update can still change it.
Getting started
New to doc-kit? CONTRIBUTING.md covers setup, running the tests and adding a changeset (node --run changeset, a patch for @doc-kit/generator-react). Leave a comment here if you'd like to take it on.
- 主要语言
- JavaScript
- 星标
- 70
- 派生
- 79
- 平均合并
- 2 天 12 小时
- 30 天内合并 PR
- 52
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
nodejs/doc-kit 的其他 Issue
-
难度 2/5 半天 新手友好度 68/100
维护者通常 3 天内回复
-
难度 2/5 1-3 小时 新手友好度 64/100
维护者通常 3 天内回复
-
难度 2/5 1-3 小时 新手友好度 55/100
nodejs/doc-kit#1194 · 1 条评论 · 1 个 reaction ·
维护者通常 3 天内回复
-
core: incremental generation against an existing output (`--incremental`)可能已有人在做 @avivkeller 于 3 天前认领。 未关闭CLI
nodejs/doc-kit#1189 · 2 条评论 · 已指派 1 人 ·
维护者通常 3 天内回复
-
cli: add a `dev` command with in-memory builds and HMR可能已有人在做 @avivkeller 于 3 天前认领。 未关闭CLI Web Generator
nodejs/doc-kit#1188 · 1 个 reaction · 已指派 1 人 ·
维护者通常 3 天内回复
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 78/100
wazuh/wazuh-documentation#10584 ·
维护者通常 1 天内回复
-
documentation good first issue
难度 1/5 1 小时以内 新手友好度 94/100
-
难度 2/5 1-3 小时 新手友好度 78/100
InukaAsith/TwinDesk#84 ·
维护者通常 2 天内回复
-
难度 2/5 1-3 小时 新手友好度 76/100
Graphify-Labs/graphify#4860 · 1 条评论 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 65/100
CommandCodeAI/command-code#1088 ·
维护者通常 2 天内回复