search_recording redoes whole-index work on every query, and the per-segment grouping is quadratic

未關閉
#346 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視

維護者通常 1 天內回覆

還沒有人認領這個 Issue。

評估

難度
4/5
預估耗時
3-5 天
新手友好度
48/100
Issue 類型
重構
描述清晰度
描述清楚
活躍度
活躍
技術堆疊
node.js, typescript

研究方向

Start with src/utils/recordingIndex.ts at the cited searchIndex and searchLibrary lines, then read src/tools/searchRecording.ts to follow the search entry point. Check how indexes are loaded and how OCR and other sources are masked before assessing the suggested grouping and caching changes. Done means multi-word search over a synthetic 50k-item index is linear and unchanged library indexes are not reparsed; the issue suggests npm test.

由索引模型根據 Issue 內容生成。

描述

effort: medium performance priority: medium

Summary

Every searchIndex call flattens every item in the index (OCR lines, events, transcript, and up to 50,000 log lines), re-masks all of them through the vm regex scan, and re-normalizes them. For a multi-word query, it then builds per-segment lists with bySeg.set(item.seg, [...(bySeg.get(item.seg) ?? []), i]), which copies the list on every item (O(k^2) per segment). With no input, library search reads and parses every *.index.json under SCREENCAST_HOME on each call. Nothing is cached, and all of it runs synchronously in the stdio server.

Where

  • src/utils/recordingIndex.ts:418-420 (re-mask per query), :436 (normalize), :441-442 (bySeg spread), :527-528 (searchLibrary)
  • src/tools/searchRecording.ts:56-60

Impact

A segment with 20k items (a burst of log lines) costs about 2e8 element copies (roughly 1 s) per multi-word query. A library of about 100 indexed recordings is re-read, re-parsed and re-masked in full on every library query. While this runs, the server answers no other request.

Suggested fix

  • Build bySeg with push, or group items by segment once.
  • OCR text is already masked on disk (#266): mask only the non-OCR sources, or store a masked copy and skip re-masking at query time.
  • Cache parsed indexes keyed by path and size/mtime, and read them with async readFile.

Acceptance criteria

  • A multi-word query over a 50k-item index runs in linear time (benchmark or test with a synthetic index).
  • Library search does not re-parse unchanged index files.

Getting started

  • Relevant files: src/utils/recordingIndex.ts, src/tools/searchRecording.ts
  • How to run tests: npm test

Filed by repo-audit/full-audit at b9b94fa. Severity: medium. Effort: M. audit-key: performance|src/utils/recordingIndex.ts|searchIndex

主要語言
TypeScript
星號
1
分支
4
平均合併
14 小時 8 分鐘
30 天內合併 PR
48

環境準備

從這裡開始

  1. 先讀完整個 Issue,再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫,在一個分支上完成修改。
  4. 送出 Pull Request,並在描述裡引用這個 Issue 編號。

TMHSDigital/screencast-mcp 的其他 Issue

查看 TMHSDigital/screencast-mcp 的全部 Issue

相似的 Issue

更多 TypeScript Issue

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。