Knowledge-Augmented Generation (KAG): evaluate schema-constrained graphs and logical-form reasoning on top of GraphRAG
I maintainer di solito rispondono entro 1 giorno
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Idoneità per principianti
- 25/100
- Tipo di issue
- Funzionalità
- Chiarezza
- Abbastanza chiara
- Stato di attività
- Attiva
- Stack tecnologico
- csharp
- Ambito
- data, machine-learning
Direzione di ricerca
Start with the ROADMAP results for Phases 5.2–5.2.2 and 6.2.1 / #405, then inspect Rag.NET.Graph, Rag.NET.GraphRag, the graph store, and the Phase 5.2 benchmark and answer-level harness. The proposed evaluation covers schema-constrained extraction, logical-form decomposition, and chunk-to-graph indexing on MultiHop-RAG. Done means benchmark results against the existing controls and a decision on whether to pursue KAG, which needs a domain-specific corpus, package placement, and OpenSPG versus native .NET implementation.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
What
Knowledge-Augmented Generation comes from Ant Group's OpenSPG framework: Liang et al., "KAG: Boosting LLMs in Professional Domains via Knowledge Augmented Generation", arXiv:2409.13731, Sep 2024. It goes beyond GraphRAG in three ways:
- An LLM-friendly, schema-constrained knowledge representation. Entities and relations follow a domain schema, rather than whatever open extraction produces.
- Mutual indexing between the knowledge graph and the source chunks, so graph facts and text evidence point at each other.
- A logical-form-guided hybrid reasoning engine. Questions are decomposed into logical forms and answered by graph traversal and symbolic steps, with text retrieval alongside.
It targets multi-hop, fact-rigorous, professional-domain questions, such as legal, medical and compliance. The cost is upfront: schema design, a heavier extraction pipeline, and keeping the graph maintained.
What Rag.NET already has
KAG would build on existing pieces, not start from nothing.
- GraphRAG:
Rag.NET.GraphandRag.NET.GraphRag, with entity extraction, a graph store, local search and global search. - The measured record (ROADMAP, Phases 5.2–5.2.2):
- On MultiHop-RAG, GraphRAG retrieval did not help: −0.02761 against the candidate-set control.
- Local search hurt answers, 0.210 against dense 0.350, because of context pollution.
- Global search helped on entity questions, 0.844 against 0.772, and did not help on yes/no questions.
- Phase 6.2.1 / #405 showed that the dropped graph endpoints are mostly common nouns and paraphrases, not entities the extractor missed.
So the open question is not "can Rag.NET do graphs". It is whether KAG's specific additions fix what the measurements showed GraphRAG getting wrong.
Proposal: measure before building
Building full KAG is a large piece of work, so this issue scopes it to an evaluation that decides whether to build it:
- Schema-constrained extraction. Run the existing extractor against a user-supplied domain schema of allowed entity and relation types. Then measure graph quality against today's open extraction on the same corpus. This tests whether a schema removes the noise behind the local-search pollution.
- Logical-form decomposition. Turn a multi-hop question into an explicit chain of graph lookups over the existing graph store, instead of a similarity walk. Measure it on the MultiHop-RAG questions that Phase 5.2 already ran.
- Chunk ↔ graph mutual index. Check what the existing graph store already records about source chunks, and what is missing for "fact → evidence chunk" citations.
Each step reuses the Phase 5.2 benchmark and answer-level harness, so the result is a number against the same controls, not a judgement call.
Needs a decision
- Whether KAG is worth pursuing at all, given that GraphRAG did not beat dense retrieval on this corpus. A professional-domain corpus with a real schema may be needed before any result means anything.
- Whether schema support goes into
Rag.NET.Graphor a new package. - Whether to integrate with OpenSPG itself, which is a Java/Python service, or reimplement the ideas natively in .NET.
Generated by Claude Code
- Lingua principale
- C#
- Stelle
- 5
- Fork
- 2
- Merge medio
- 14h 28m
- PR unite (30g)
- 190
Preparare l'ambiente
- Nessun Dockerfile né file Docker Compose
- Ha un modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di MarcelRoozekrans/Rag.NET
-
bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
MarcelRoozekrans/Rag.NET#1030 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 5/5 Più di una settimana Idoneità per principianti 35/100
MarcelRoozekrans/Rag.NET#944 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 35/100
MarcelRoozekrans/Rag.NET#937 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 48/100
MarcelRoozekrans/Rag.NET#888 ·
I maintainer di solito rispondono entro 1 giorno
-
enhancement
Difficoltà 5/5 Più di una settimana Idoneità per principianti 25/100
MarcelRoozekrans/Rag.NET#854 ·
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di MarcelRoozekrans/Rag.NET
Issue simili
-
quality 提案 要トリアージ
Difficoltà 2/5 1-3 ore Idoneità per principianti 64/100
I maintainer di solito rispondono entro 1 giorno
-
The Language and Region lists in Options read to a screen reader as "LanguageInfo { Id = ar, ... }"Apertabug localization small ui
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
hexide-io/HexIDE#4176 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
ReapplyLocalizationsToAllComponents aborts (and keeps aborting) on a disposed registered controlAperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 88/100
AvaloniaUI/Avalonia#22841 ·
I maintainer di solito rispondono entro 1 giorno
-
a11y tech-debt
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 78/100
philipphoeninger/procure-access#350 ·
I maintainer di solito rispondono entro 1 giorno