epic: Declarative Link-Driven Knowledge Base & Granular Incremental Indexing
Maintainers usually reply within 1 day
@DerekRoberts is already working on this.
Since Sep 21, 2026.
Assessment
This issue has not been assessed yet.
Description
Overview
Transition the Agreement Navigator knowledge base from a repository of committed static Markdown files and binary PDFs to a declarative link-driven library (sources.yaml) with granular per-document caching and incremental FAISS re-indexing.
Core Invariants & Goals
- Gradual & Safe: No committed files deleted without proven 100% substantive parity via shadow audits.
- Incremental Rebuilds: Document modifications re-embed only the changed files, keeping FAISS re-indexing under <50ms.
- Network Efficiency: Conditional HTTP requests (
ETag/304 Not Modified) prevent unnecessary downloads during scheduled drift runs.
Child Issues
- #697: feat(indexing): Implement per-document chunk and embedding caching (.npy) for incremental FAISS builds
- #698: feat(ingestion): Support conditional HTTP headers (ETag / If-Modified-Since) and 304 skipping in sync_sources.py
- #699: feat(ingestion): Add PDF extraction strategy and shadow-audit multi-part BCLaws statutes into sources.yaml
Architectural Contract
Documented in app/SPEC.md (Section 6: Forensic Markdown Pipeline & Section 12: Future Roadmap).
- Dominant language
- Python
- Stars
- 2
- Forks
- 1
- Avg merge
- 4h 16m
- Merged PRs (30d)
- 38
Getting set up
- Ships a Dockerfile or Docker Compose file
- No pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from MinionTech/vexilon
-
Difficulty 2/5 Half a day Newbie friendliness 66/100
MinionTech/vexilon#844 ·
Maintainers usually reply within 1 day
-
OHS Part 8 source missing B.C. Reg. 135/2026 (face-seal amendments, in force 2026-11-06)Possibly taken @DerekRoberts claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 55/100
MinionTech/vexilon#842 · 1 assignee ·
Maintainers usually reply within 1 day
-
research: consume from canli.orgPossibly taken @DerekRoberts claimed this 1 day ago. Open
MinionTech/vexilon#836 · 1 assignee ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 5/5 Over a week Newbie friendliness 22/100
MinionTech/vexilon#821 · 1 assignee ·
Maintainers usually reply within 1 day
-
feat(rag): hybrid BM25 + FAISS retrieval with reciprocal-rank fusionMay be free again @DerekRoberts claimed this 28 days ago, and no pull request is open. Openenhancement
MinionTech/vexilon#802 · 1 assignee ·
Maintainers usually reply within 1 day
All issues in MinionTech/vexilon
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 64/100
cal-itp/data-infra#5887 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
ranacseruet/vaice#139 ·
Maintainers usually reply within 1 day
-
numpy.bytes_ attributes are recorded as a truncated repr instead of their contentPossibly taken @JoeyTan21 claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
batocera-linux/batocera.linux#17423 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100