Adding a source to an existing index is unverifiable: no corpus audit, no staging path, no before/after baseline
メンテナーはふだん 1 日以内に返信
まだ誰も着手していません。
評価
- 難易度
- 5/5
- 見積もり時間
- 1週間以上
- 初心者へのやさしさ
- 28/100
調査の方向性
No implementation files or tests are named. First read the existing IRagDataManager, evaluation, shadow-pipeline, and PgVectorStore surfaces referenced in the issue, then discuss which of the sketched directions should be proposed. Done is not defined yet: the issue explicitly invites design decisions about scope and API ownership.
索引モデルが issue の本文から書いたものです。
説明
Adding a source to an index that already holds a corpus is currently an unverifiable operation: nothing in the library tells you what the new data did to the old data, before or after it lands.
The question came from a real one — "we ingest from different sources; how do we confirm the ingested data is correct?", then "I already have a big index and want to add a source without wrecking it." The answer today is a list of parts, not a path.
What already exists, and what it does not cover
| Piece | Covers | Does not cover |
|---|---|---|
IRagDataManager.GetDocumentsAsync / GetChunksAsync / GetStatsAsync |
reading back what was ingested, per document | no filtering — by tag, by source, by ingest time; a big corpus comes back whole or one id at a time |
EvaluationDatasetBuilder + RagasEvaluationSuite + RagAbTester |
measuring answer quality, with confidence intervals | requires you to assemble the before/after harness yourself |
UseShadow<TSecondary> |
a candidate pipeline against live traffic | needs a second fully-built pipeline; nothing helps you construct the staging one |
Rag.NET.Diagnostics traces |
per-query chunk scores by stage | one query at a time, after the fact |
UseEmbeddingVersioning |
detecting a model change per document | only fires when the model identity string changes |
ragnet CLI |
ingest, query |
no inspect, no audit, no diff |
The three ways a new source damages an existing index
All three are silent today.
- Document id collisions. Store is an upsert on
(document_id, chunk_index). A new source reusing an existing id withoutIngestionOptions.Overwritereplaces the first N chunks and leaves every chunk past the new length in place — one document assembled from two sources. - An incompatible embedder. Two models at the same dimension count are structurally interchangeable and semantically incomparable.
UseEmbeddingVersioningcatches a changed identity string; it cannot catch a deployment repointed at a new model behind an unchanged name, which is the Foundry case documented in #663. - Dilution. The new source is extracted correctly and still degrades retrieval by winning top-k from better chunks. Only measurable, and only against a baseline taken before the ingest.
Gaps this would close
- No way to list or delete by source.
DeleteAsyncis per document id,RetrievalOptions.MetadataFilteris equality-only — so a single index cannot answer "what would retrieval return without this source?", and rollback depends on having recorded every id yourself. - No staging path.
PgVectorStorereads and writes a fixedrag_chunkstable, so an isolated trial index means a second connection string and a secondServiceProvider;CreateCollectionAsyncmakes tables the pipeline never queries. - No corpus audit. Chunk-length distribution, empty or near-empty extractions, near-duplicate chunks, per-source counts — the signals that say a PDF OCR'd into soup or an HTML crawl ingested nav chrome — are computable from the sidecar store and computed by nothing.
Not yet a proposal
Sketching only, to give the discussion something to disagree with:
- an audit surface over
IRagDataManager— filter by tag, distributions, duplicate detection, per-source counts — plusragnet auditover it; - a documented stage-and-promote recipe, with whatever the library needs to make a second index cheap to stand up;
- a before/after harness that wraps
EvaluationDatasetBuilderandRagAbTesterinto one call, since the baseline has to be taken before the change and that is the step people skip.
@StefH — worth your opinion, specifically
- Does stage-then-promote belong in the library at all, or is it an application concern the docs should describe and nothing more? I lean toward a recipe plus small primitives, but that may be the wrong instinct.
- If
IRagDataManagergrows a filtered read surface, is that the right home, or does an audit live outside the interface reading the sidecar directly? - Delete-by-metadata-filter across seven stores — plausible to make uniform, or an invitation to seven subtly different semantics? Today delete is per document id everywhere, which is at least consistent.
- Anything from your own work that solved the shape "validate a change against production data before it reaches production"? That is the part I would rather borrow than invent.
- 主要言語
- C#
- スター
- 5
- フォーク
- 2
- 平均マージ
- 14時間 28分
- マージ済み PR(30日)
- 190
環境構築
- Dockerfile・Docker Compose ファイルなし
- プルリクエストのテンプレートあり
- コントリビューションガイドを読む
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
MarcelRoozekrans/Rag.NET のほかの issue
-
bug
難易度 2/5 1〜3時間 初心者へのやさしさ 68/100
MarcelRoozekrans/Rag.NET#1030 ·
メンテナーはふだん 1 日以内に返信
-
難易度 5/5 1週間以上 初心者へのやさしさ 35/100
MarcelRoozekrans/Rag.NET#944 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
難易度 4/5 3〜5日 初心者へのやさしさ 35/100
MarcelRoozekrans/Rag.NET#937 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
難易度 4/5 3〜5日 初心者へのやさしさ 48/100
MarcelRoozekrans/Rag.NET#888 ·
メンテナーはふだん 1 日以内に返信
-
enhancement
難易度 5/5 1週間以上 初心者へのやさしさ 25/100
MarcelRoozekrans/Rag.NET#855 ·
メンテナーはふだん 1 日以内に返信
MarcelRoozekrans/Rag.NET の issue をすべて見る
似ている issue
-
quality 提案 要トリアージ
難易度 2/5 1〜3時間 初心者へのやさしさ 64/100
メンテナーはふだん 1 日以内に返信
-
The Language and Region lists in Options read to a screen reader as "LanguageInfo { Id = ar, ... }"オープンbug localization small ui
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
hexide-io/HexIDE#4176 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
ReapplyLocalizationsToAllComponents aborts (and keeps aborting) on a disposed registered controlオープン
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
-
難易度 2/5 1〜3時間 初心者へのやさしさ 88/100
AvaloniaUI/Avalonia#22841 ·
メンテナーはふだん 1 日以内に返信
-
a11y tech-debt
難易度 1/5 1時間未満 初心者へのやさしさ 78/100
philipphoeninger/procure-access#350 ·
メンテナーはふだん 1 日以内に返信