Knowledge-Augmented Generation (KAG): evaluate schema-constrained graphs and logical-form reasoning on top of GraphRAG
Los mantenedores suelen responder en 1 día
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 5/5
- Tiempo estimado
- Más de una semana
- Aptitud para principiantes
- 25/100
- Tipo de issue
- Nueva funcionalidad
- Claridad
- Bastante claro
- Estado de actividad
- Activo
- Stack tecnológico
- csharp
- Área
- data, machine-learning
Línea de trabajo
Start with the ROADMAP results for Phases 5.2–5.2.2 and 6.2.1 / #405, then inspect Rag.NET.Graph, Rag.NET.GraphRag, the graph store, and the Phase 5.2 benchmark and answer-level harness. The proposed evaluation covers schema-constrained extraction, logical-form decomposition, and chunk-to-graph indexing on MultiHop-RAG. Done means benchmark results against the existing controls and a decision on whether to pursue KAG, which needs a domain-specific corpus, package placement, and OpenSPG versus native .NET implementation.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
What
Knowledge-Augmented Generation comes from Ant Group's OpenSPG framework: Liang et al., "KAG: Boosting LLMs in Professional Domains via Knowledge Augmented Generation", arXiv:2409.13731, Sep 2024. It goes beyond GraphRAG in three ways:
- An LLM-friendly, schema-constrained knowledge representation. Entities and relations follow a domain schema, rather than whatever open extraction produces.
- Mutual indexing between the knowledge graph and the source chunks, so graph facts and text evidence point at each other.
- A logical-form-guided hybrid reasoning engine. Questions are decomposed into logical forms and answered by graph traversal and symbolic steps, with text retrieval alongside.
It targets multi-hop, fact-rigorous, professional-domain questions, such as legal, medical and compliance. The cost is upfront: schema design, a heavier extraction pipeline, and keeping the graph maintained.
What Rag.NET already has
KAG would build on existing pieces, not start from nothing.
- GraphRAG:
Rag.NET.GraphandRag.NET.GraphRag, with entity extraction, a graph store, local search and global search. - The measured record (ROADMAP, Phases 5.2–5.2.2):
- On MultiHop-RAG, GraphRAG retrieval did not help: −0.02761 against the candidate-set control.
- Local search hurt answers, 0.210 against dense 0.350, because of context pollution.
- Global search helped on entity questions, 0.844 against 0.772, and did not help on yes/no questions.
- Phase 6.2.1 / #405 showed that the dropped graph endpoints are mostly common nouns and paraphrases, not entities the extractor missed.
So the open question is not "can Rag.NET do graphs". It is whether KAG's specific additions fix what the measurements showed GraphRAG getting wrong.
Proposal: measure before building
Building full KAG is a large piece of work, so this issue scopes it to an evaluation that decides whether to build it:
- Schema-constrained extraction. Run the existing extractor against a user-supplied domain schema of allowed entity and relation types. Then measure graph quality against today's open extraction on the same corpus. This tests whether a schema removes the noise behind the local-search pollution.
- Logical-form decomposition. Turn a multi-hop question into an explicit chain of graph lookups over the existing graph store, instead of a similarity walk. Measure it on the MultiHop-RAG questions that Phase 5.2 already ran.
- Chunk ↔ graph mutual index. Check what the existing graph store already records about source chunks, and what is missing for "fact → evidence chunk" citations.
Each step reuses the Phase 5.2 benchmark and answer-level harness, so the result is a number against the same controls, not a judgement call.
Needs a decision
- Whether KAG is worth pursuing at all, given that GraphRAG did not beat dense retrieval on this corpus. A professional-domain corpus with a real schema may be needed before any result means anything.
- Whether schema support goes into
Rag.NET.Graphor a new package. - Whether to integrate with OpenSPG itself, which is a Java/Python service, or reimplement the ideas natively in .NET.
Generated by Claude Code
- Lenguaje dominante
- C#
- Estrellas
- 5
- Forks
- 2
- Merge medio
- 14 h 28 min
- PR fusionados (30 d)
- 190
Preparar el entorno
- Sin Dockerfile ni archivo de Docker Compose
- Tiene una plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de MarcelRoozekrans/Rag.NET
-
bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
MarcelRoozekrans/Rag.NET#1030 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 5/5 Más de una semana Aptitud para principiantes 35/100
MarcelRoozekrans/Rag.NET#944 · 1 comentario ·
Los mantenedores suelen responder en 1 día
-
Dificultad 4/5 3-5 días Aptitud para principiantes 35/100
MarcelRoozekrans/Rag.NET#937 · 1 comentario ·
Los mantenedores suelen responder en 1 día
-
Dificultad 4/5 3-5 días Aptitud para principiantes 48/100
MarcelRoozekrans/Rag.NET#888 ·
Los mantenedores suelen responder en 1 día
-
enhancement
Dificultad 5/5 Más de una semana Aptitud para principiantes 25/100
MarcelRoozekrans/Rag.NET#854 ·
Los mantenedores suelen responder en 1 día
Todos los issues de MarcelRoozekrans/Rag.NET
Issues similares
-
quality 提案 要トリアージ
Dificultad 2/5 1-3 horas Aptitud para principiantes 64/100
Los mantenedores suelen responder en 1 día
-
The Language and Region lists in Options read to a screen reader as "LanguageInfo { Id = ar, ... }"Abiertobug localization small ui
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
hexide-io/HexIDE#4176 · 1 comentario ·
Los mantenedores suelen responder en 1 día
-
ReapplyLocalizationsToAllComponents aborts (and keeps aborting) on a disposed registered controlAbierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 88/100
AvaloniaUI/Avalonia#22841 ·
Los mantenedores suelen responder en 1 día
-
a11y tech-debt
Dificultad 1/5 Menos de una hora Aptitud para principiantes 78/100
philipphoeninger/procure-access#350 ·
Los mantenedores suelen responder en 1 día