Knowledge-Augmented Generation (KAG): evaluate schema-constrained graphs and logical-form reasoning on top of GraphRAG

Abierto
#855 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
25/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
csharp

Línea de trabajo

Start with the ROADMAP results for Phases 5.2–5.2.2 and 6.2.1 / #405, then inspect Rag.NET.Graph, Rag.NET.GraphRag, the graph store, and the Phase 5.2 benchmark and answer-level harness. The proposed evaluation covers schema-constrained extraction, logical-form decomposition, and chunk-to-graph indexing on MultiHop-RAG. Done means benchmark results against the existing controls and a decision on whether to pursue KAG, which needs a domain-specific corpus, package placement, and OpenSPG versus native .NET implementation.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

enhancement

What

Knowledge-Augmented Generation comes from Ant Group's OpenSPG framework: Liang et al., "KAG: Boosting LLMs in Professional Domains via Knowledge Augmented Generation", arXiv:2409.13731, Sep 2024. It goes beyond GraphRAG in three ways:

  1. An LLM-friendly, schema-constrained knowledge representation. Entities and relations follow a domain schema, rather than whatever open extraction produces.
  2. Mutual indexing between the knowledge graph and the source chunks, so graph facts and text evidence point at each other.
  3. A logical-form-guided hybrid reasoning engine. Questions are decomposed into logical forms and answered by graph traversal and symbolic steps, with text retrieval alongside.

It targets multi-hop, fact-rigorous, professional-domain questions, such as legal, medical and compliance. The cost is upfront: schema design, a heavier extraction pipeline, and keeping the graph maintained.

What Rag.NET already has

KAG would build on existing pieces, not start from nothing.

  • GraphRAG: Rag.NET.Graph and Rag.NET.GraphRag, with entity extraction, a graph store, local search and global search.
  • The measured record (ROADMAP, Phases 5.2–5.2.2):
    • On MultiHop-RAG, GraphRAG retrieval did not help: −0.02761 against the candidate-set control.
    • Local search hurt answers, 0.210 against dense 0.350, because of context pollution.
    • Global search helped on entity questions, 0.844 against 0.772, and did not help on yes/no questions.
    • Phase 6.2.1 / #405 showed that the dropped graph endpoints are mostly common nouns and paraphrases, not entities the extractor missed.

So the open question is not "can Rag.NET do graphs". It is whether KAG's specific additions fix what the measurements showed GraphRAG getting wrong.

Proposal: measure before building

Building full KAG is a large piece of work, so this issue scopes it to an evaluation that decides whether to build it:

  1. Schema-constrained extraction. Run the existing extractor against a user-supplied domain schema of allowed entity and relation types. Then measure graph quality against today's open extraction on the same corpus. This tests whether a schema removes the noise behind the local-search pollution.
  2. Logical-form decomposition. Turn a multi-hop question into an explicit chain of graph lookups over the existing graph store, instead of a similarity walk. Measure it on the MultiHop-RAG questions that Phase 5.2 already ran.
  3. Chunk ↔ graph mutual index. Check what the existing graph store already records about source chunks, and what is missing for "fact → evidence chunk" citations.

Each step reuses the Phase 5.2 benchmark and answer-level harness, so the result is a number against the same controls, not a judgement call.

Needs a decision

  • Whether KAG is worth pursuing at all, given that GraphRAG did not beat dense retrieval on this corpus. A professional-domain corpus with a real schema may be needed before any result means anything.
  • Whether schema support goes into Rag.NET.Graph or a new package.
  • Whether to integrate with OpenSPG itself, which is a Java/Python service, or reimplement the ideas natively in .NET.

Generated by Claude Code

Lenguaje dominante
C#
Estrellas
5
Forks
2
Merge medio
14 h 28 min
PR fusionados (30 d)
190

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de MarcelRoozekrans/Rag.NET

Todos los issues de MarcelRoozekrans/Rag.NET

Issues similares

Más issues de C#

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.