The bundled embedding model is FP16, not the int8 ADR 0007 specifies

オープン
  • 何か問題がありますか？
#1,996 コメント 4 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

@ybai08 がすでに取り組んでいます。

2026年10月2日 から。

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
52/100
issue の種類
機能追加
明瞭さ
明確に書かれている
活発さ
活発
技術スタック
python
領域
machine-learning

調査の方向性

Read embedding.py, ADR 0007, and the retrieval proposal, then inspect run_retrieval_eval.py and run its evaluation against both models. Measure file size, CPU throughput, vector cosine similarity, and per-slice retrieval ranks; post the evidence and re-embedding recommendation on the issue before opening a PR. Done means the ADR and code match measured results, the Known gaps entry is removed, and the unchanged integration tests pass.

索引モデルが issue の本文から書いたものです。

説明

architecture desktop help wanted performance

The gap

docs/architecture/search.md lists this under Known gaps:

The bundled model is FP16, not the int8 ADR 0007 specifies: Qdrant/bge-small-en-v1.5-onnx-Q is named for quantization but its only ONNX file stores FLOAT16 weights. An int8 export would roughly halve the 63 MB and speed embedding, and needs checking for whether its vectors are close enough to skip a re-embed.

Why it matters

ADR 0007 says "bge-small-en-v1.5, int8 ONNX, 384 dimensions, about 66 MB", and the comment at the top of embedding.py repeats it. Neither is true: the repository is named for quantization and its one ONNX file holds FLOAT16 weights. The ADR is the document a reader trusts about what runs on the user's machine, so an ADR that is wrong about the artefact is worse than a missing one.

The cost is real as well as documentary. The model is bundled into every installer for three platforms, and embedding runs on the CPU in the ingest worker, where it is the slow half of a big import and competes with the parser. The retrieval proposal puts an int8 bge at about 33 MB against today's 63 MB, and it cites that figure when comparing bundle sizes for the multilingual swap — so the comparison in that proposal is resting on a number nobody has produced.

Where

What done looks like

This is an investigation with a code change at the end, not the other way round. Expect most of the work to be measurement.

  • The ADR and the code agree about what is bundled, and the numbers in the ADR are ones you measured: file size on disk, and embedding throughput on a CPU you name.
  • A decision, with the evidence behind it, on whether an int8 export's vectors are close enough to the FP16 model's to leave existing indexes alone. Cosine between the two models' vectors for the same passage, and the eval's per-slice ranks under each, are the two things that answer it. The threshold is yours to set and to defend.
  • If the answer is that a re-embed is needed, stop and say so on this issue. Then this stops being a small win and becomes part of the migration the retrieval proposal already designs, which is maintainer work — closing the ADR's documentary half is still worth a PR on its own.
  • Delete the Known gaps line from docs/architecture/search.md in the same PR as whichever half you land, and fix the Code line's claim if it changes.

What not to change

Do not quietly swap the model. _check_embedding_width() in shared/migrations.py refuses a database whose chunk_vectors width no longer matches the setting, and both ADR 0007 and the retrieval proposal say in as many words that a same-width replacement is undetectable: an existing library would go on searching vectors written by a different model, silently, with no error and slowly worsening results. An int8 bge is 384 dimensions, so that guard will not fire for you.

How to verify

surfsense_local/backend/tests/integration/search/test_retrieve.py covers keyword, paraphrase and Hindi retrieval; tests/integration/chunks/test_search_index.py covers the index's refusal of a vector of the wrong width. Both should pass unchanged — if one fails, the swap is not a swap. The eval is not a test and is run by hand.

cd surfsense_local/backend
uv run pytest -m unit
uv run pytest -m integration
uv run ruff check .

Before you start

Read CONTRIBUTING.md, the search doc, ADR 0007 and the retrieval proposal, which is the accepted design for changing the embedder and explains why the re-embed is the expensive part. Post your measurements on this issue before you open the PR — the numbers are the contribution here, and a maintainer will want to see them first. Open your PR against dev, not main. Comment here to claim this.

主要言語
Python
スター
16.3k
フォーク
1.6k
平均マージ
14時間 22分
マージ済み PR（30日）
175

環境構築

このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

MODSetter/SurfSense のほかの issue

MODSetter/SurfSense の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。