Adding a source to an existing index is unverifiable: no corpus audit, no staging path, no before/after baseline
I maintainer di solito rispondono entro 1 giorno
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Idoneità per principianti
- 28/100
Direzione di ricerca
No implementation files or tests are named. First read the existing IRagDataManager, evaluation, shadow-pipeline, and PgVectorStore surfaces referenced in the issue, then discuss which of the sketched directions should be proposed. Done is not defined yet: the issue explicitly invites design decisions about scope and API ownership.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Adding a source to an index that already holds a corpus is currently an unverifiable operation: nothing in the library tells you what the new data did to the old data, before or after it lands.
The question came from a real one — "we ingest from different sources; how do we confirm the ingested data is correct?", then "I already have a big index and want to add a source without wrecking it." The answer today is a list of parts, not a path.
What already exists, and what it does not cover
| Piece | Covers | Does not cover |
|---|---|---|
IRagDataManager.GetDocumentsAsync / GetChunksAsync / GetStatsAsync |
reading back what was ingested, per document | no filtering — by tag, by source, by ingest time; a big corpus comes back whole or one id at a time |
EvaluationDatasetBuilder + RagasEvaluationSuite + RagAbTester |
measuring answer quality, with confidence intervals | requires you to assemble the before/after harness yourself |
UseShadow<TSecondary> |
a candidate pipeline against live traffic | needs a second fully-built pipeline; nothing helps you construct the staging one |
Rag.NET.Diagnostics traces |
per-query chunk scores by stage | one query at a time, after the fact |
UseEmbeddingVersioning |
detecting a model change per document | only fires when the model identity string changes |
ragnet CLI |
ingest, query |
no inspect, no audit, no diff |
The three ways a new source damages an existing index
All three are silent today.
- Document id collisions. Store is an upsert on
(document_id, chunk_index). A new source reusing an existing id withoutIngestionOptions.Overwritereplaces the first N chunks and leaves every chunk past the new length in place — one document assembled from two sources. - An incompatible embedder. Two models at the same dimension count are structurally interchangeable and semantically incomparable.
UseEmbeddingVersioningcatches a changed identity string; it cannot catch a deployment repointed at a new model behind an unchanged name, which is the Foundry case documented in #663. - Dilution. The new source is extracted correctly and still degrades retrieval by winning top-k from better chunks. Only measurable, and only against a baseline taken before the ingest.
Gaps this would close
- No way to list or delete by source.
DeleteAsyncis per document id,RetrievalOptions.MetadataFilteris equality-only — so a single index cannot answer "what would retrieval return without this source?", and rollback depends on having recorded every id yourself. - No staging path.
PgVectorStorereads and writes a fixedrag_chunkstable, so an isolated trial index means a second connection string and a secondServiceProvider;CreateCollectionAsyncmakes tables the pipeline never queries. - No corpus audit. Chunk-length distribution, empty or near-empty extractions, near-duplicate chunks, per-source counts — the signals that say a PDF OCR'd into soup or an HTML crawl ingested nav chrome — are computable from the sidecar store and computed by nothing.
Not yet a proposal
Sketching only, to give the discussion something to disagree with:
- an audit surface over
IRagDataManager— filter by tag, distributions, duplicate detection, per-source counts — plusragnet auditover it; - a documented stage-and-promote recipe, with whatever the library needs to make a second index cheap to stand up;
- a before/after harness that wraps
EvaluationDatasetBuilderandRagAbTesterinto one call, since the baseline has to be taken before the change and that is the step people skip.
@StefH — worth your opinion, specifically
- Does stage-then-promote belong in the library at all, or is it an application concern the docs should describe and nothing more? I lean toward a recipe plus small primitives, but that may be the wrong instinct.
- If
IRagDataManagergrows a filtered read surface, is that the right home, or does an audit live outside the interface reading the sidecar directly? - Delete-by-metadata-filter across seven stores — plausible to make uniform, or an invitation to seven subtly different semantics? Today delete is per document id everywhere, which is at least consistent.
- Anything from your own work that solved the shape "validate a change against production data before it reaches production"? That is the part I would rather borrow than invent.
- Lingua principale
- C#
- Stelle
- 5
- Fork
- 2
- Merge medio
- 14h 28m
- PR unite (30g)
- 190
Preparare l'ambiente
- Nessun Dockerfile né file Docker Compose
- Ha un modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di MarcelRoozekrans/Rag.NET
-
bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
MarcelRoozekrans/Rag.NET#1030 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 5/5 Più di una settimana Idoneità per principianti 35/100
MarcelRoozekrans/Rag.NET#944 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 35/100
MarcelRoozekrans/Rag.NET#937 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 48/100
MarcelRoozekrans/Rag.NET#888 ·
I maintainer di solito rispondono entro 1 giorno
-
enhancement
Difficoltà 5/5 Più di una settimana Idoneità per principianti 25/100
MarcelRoozekrans/Rag.NET#855 ·
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di MarcelRoozekrans/Rag.NET
Issue simili
-
quality 提案 要トリアージ
Difficoltà 2/5 1-3 ore Idoneità per principianti 64/100
I maintainer di solito rispondono entro 1 giorno
-
The Language and Region lists in Options read to a screen reader as "LanguageInfo { Id = ar, ... }"Apertabug localization small ui
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
hexide-io/HexIDE#4176 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
ReapplyLocalizationsToAllComponents aborts (and keeps aborting) on a disposed registered controlAperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 88/100
AvaloniaUI/Avalonia#22841 ·
I maintainer di solito rispondono entro 1 giorno
-
a11y tech-debt
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 78/100
philipphoeninger/procure-access#350 ·
I maintainer di solito rispondono entro 1 giorno