Hacktoberfest 2026: những issue maintainer đã đánh dấu cho tháng Mười, đang mở và phù hợp người mới. Xem issue Hacktoberfest

feat: documents contribute to the knowledge graph

Đang mở
  • Có gì không ổn?
#235 1 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức phù hợp với người mới
28/100
Loại issue
Tính năng
Độ rõ ràng
Khá rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
dart
Lĩnh vực
ai

Hướng nghiên cứu

Start by reading ExtractKnowledgeUseCase, InferenceScheduler, PromptBudget, and RagPipeline to understand the existing note extraction and retrieval paths. Work out how document identity and long inputs should be handled before implementation; the issue leaves both open. Done means a document is extracted once per document, participates in graph expansion, and long inputs are handled without silent truncation.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

enhancement P2-Medium P3-Low

Notes participate in a knowledge graph — e tags are edges, t tags are topic nodes, and ExtractKnowledgeUseCase pulls entities and relations that feed graph expansion during retrieval. A document's chunks participate in none of it. They are retrieved by vector similarity alone, then explicitly skip graph expansion and memory lookup, because a chunk is not a graph node (RagPipeline).

So a PDF that is entirely about a topic the user has notes on contributes nothing to the graph around that topic, and a question answered from the graph never reaches the document.

What to build

Let a document contribute to the graph — one extraction pass per document, not per chunk.

Per-chunk extraction is the wrong shape and worth stating plainly: a long PDF is dozens of chunks, each pass is an LLM call on a device already contending for the model with chat and Gana, and chunk boundaries cut mid-argument so the extracted relations would be worse than from whole text. Document-level extraction is one call over the joined text, which is also what the existing note path already does.

Pointers:

  • ExtractKnowledgeUseCase is the existing extraction path; the question is whether it generalises or needs a document-shaped sibling
  • InferenceScheduler has a T2 extract tier — document extraction belongs there, not on the chat path
  • PromptBudget caps what one call can carry; a long document will not fit in one pass on the smallest local model, so this needs a real answer for long inputs rather than silent truncation

Open questions

  • What is a document's node identity in the graph? The attaching note, or the document itself? A document can be attached to more than one note.
  • Does a not-searchable scan get any graph presence at all?

Part of #228 · Feature it follows from: #226

Status: backlog (2026-10-13)

Not feasible to do well with on-device compute today, so this is parked. The reasoning is in the comment below; the short version: local models cap one call at 1k-8k tokens (PromptBudget), a PDF is 10k-100k+, and every workaround (truncate, per-chunk calls, map-reduce) is either wrong or too expensive on a phone that is already contended. Revisit when local context and throughput improve, or when UNIUN Cloud is the default for extraction.

Options to consider later (no LLM call, not built)
  1. Attachment edge. A document is already attached to a note. Treat that as a graph edge so expansion reaching the note brings its document's top chunks. Zero cost; overlaps with #236.
  2. Vector-derived related edges. At index time compare a document's chunk vectors (or centroid) to existing note vectors and store the top matches as note-document edges. A few dot products, no model call.
  3. Non-LLM topic nodes. RAKE or TF-IDF keyphrases over the already-extracted text become topic nodes, the way t tags do for notes.
  4. Cloud-only extraction, opt-in. A single full-document pass is realistic only with a large cloud context window. Gate on LlmBackendType.uniunCloud; needs a privacy note since document text leaves the device.

Still to answer before building any of these: is a document's node identity the attaching note or the document itself (it can be attached to several notes), and does a not-searchable scan get any graph presence?

Ngôn ngữ chính
Dart
Star
2
Fork
2
Merge trung bình
1 ngày 35 phút
Pull request đã merge (30 ngày)
36

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của basictech01/uniun

Tất cả issue của basictech01/uniun

Issue tương tự

Thêm issue về Dart

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.