Knowledge-Augmented Generation (KAG): evaluate schema-constrained graphs and logical-form reasoning on top of GraphRAG

Đang mở
#855 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức phù hợp với người mới
25/100
Loại issue
Tính năng
Độ rõ ràng
Khá rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
csharp
Lĩnh vực
data, machine-learning

Hướng nghiên cứu

Start with the ROADMAP results for Phases 5.2–5.2.2 and 6.2.1 / #405, then inspect Rag.NET.Graph, Rag.NET.GraphRag, the graph store, and the Phase 5.2 benchmark and answer-level harness. The proposed evaluation covers schema-constrained extraction, logical-form decomposition, and chunk-to-graph indexing on MultiHop-RAG. Done means benchmark results against the existing controls and a decision on whether to pursue KAG, which needs a domain-specific corpus, package placement, and OpenSPG versus native .NET implementation.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

enhancement

What

Knowledge-Augmented Generation comes from Ant Group's OpenSPG framework: Liang et al., "KAG: Boosting LLMs in Professional Domains via Knowledge Augmented Generation", arXiv:2409.13731, Sep 2024. It goes beyond GraphRAG in three ways:

  1. An LLM-friendly, schema-constrained knowledge representation. Entities and relations follow a domain schema, rather than whatever open extraction produces.
  2. Mutual indexing between the knowledge graph and the source chunks, so graph facts and text evidence point at each other.
  3. A logical-form-guided hybrid reasoning engine. Questions are decomposed into logical forms and answered by graph traversal and symbolic steps, with text retrieval alongside.

It targets multi-hop, fact-rigorous, professional-domain questions, such as legal, medical and compliance. The cost is upfront: schema design, a heavier extraction pipeline, and keeping the graph maintained.

What Rag.NET already has

KAG would build on existing pieces, not start from nothing.

  • GraphRAG: Rag.NET.Graph and Rag.NET.GraphRag, with entity extraction, a graph store, local search and global search.
  • The measured record (ROADMAP, Phases 5.2–5.2.2):
    • On MultiHop-RAG, GraphRAG retrieval did not help: −0.02761 against the candidate-set control.
    • Local search hurt answers, 0.210 against dense 0.350, because of context pollution.
    • Global search helped on entity questions, 0.844 against 0.772, and did not help on yes/no questions.
    • Phase 6.2.1 / #405 showed that the dropped graph endpoints are mostly common nouns and paraphrases, not entities the extractor missed.

So the open question is not "can Rag.NET do graphs". It is whether KAG's specific additions fix what the measurements showed GraphRAG getting wrong.

Proposal: measure before building

Building full KAG is a large piece of work, so this issue scopes it to an evaluation that decides whether to build it:

  1. Schema-constrained extraction. Run the existing extractor against a user-supplied domain schema of allowed entity and relation types. Then measure graph quality against today's open extraction on the same corpus. This tests whether a schema removes the noise behind the local-search pollution.
  2. Logical-form decomposition. Turn a multi-hop question into an explicit chain of graph lookups over the existing graph store, instead of a similarity walk. Measure it on the MultiHop-RAG questions that Phase 5.2 already ran.
  3. Chunk ↔ graph mutual index. Check what the existing graph store already records about source chunks, and what is missing for "fact → evidence chunk" citations.

Each step reuses the Phase 5.2 benchmark and answer-level harness, so the result is a number against the same controls, not a judgement call.

Needs a decision

  • Whether KAG is worth pursuing at all, given that GraphRAG did not beat dense retrieval on this corpus. A professional-domain corpus with a real schema may be needed before any result means anything.
  • Whether schema support goes into Rag.NET.Graph or a new package.
  • Whether to integrate with OpenSPG itself, which is a Java/Python service, or reimplement the ideas natively in .NET.

Generated by Claude Code

Ngôn ngữ chính
C#
Star
5
Fork
2
Merge trung bình
14 giờ 28 phút
Pull request đã merge (30 ngày)
190

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của MarcelRoozekrans/Rag.NET

Tất cả issue của MarcelRoozekrans/Rag.NET

Issue tương tự

Thêm issue về C#

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.