Ingest several sources in parallel
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 72/100
Research direction
Start with IngestionManager.process_pending() in gknowledge/core/ingestion.py and run make test to understand the offline tests. Implement configurable concurrent processing for 2–3 sources while keeping offline test behavior deterministic. Confirm the index and overview are rebuilt once after processing completes.
Written by the indexing model from the issue text.
Description
With ChatGPT, ingestion takes about 25 s per short source, one at a time.
Where to look: IngestionManager.process_pending() in gknowledge/core/ingestion.py. The SQLite store is already safe across threads.
Done when: 2–3 sources are processed concurrently (configurable), the index and overview are rebuilt once at the end, and offline mode stays deterministic in tests.
New here? Run make start to open a demo project that restarts on every save, and make test for the core tests. See CONTRIBUTING.md. Comment below to claim the issue.
- Dominant language
- Python
- Stars
- 13
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from openapi/gknowledge
-
area: agents area: docs good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
openapi/gknowledge#23 ·
-
Add an EPUB parserOpenarea: core good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
openapi/gknowledge#21 ·
-
area: ui good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
openapi/gknowledge#19 ·
-
area: ui good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
openapi/gknowledge#13 ·
-
area: ui good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
openapi/gknowledge#11 ·
All issues in openapi/gknowledge
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 1/5 Under an hour Newbie friendliness 62/100
-
Difficulty 1/5 Under an hour Newbie friendliness 70/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
topoteretes/cognee#5773 ·
Maintainers usually reply within 1 day
-
MiKTeX: a failed build reports no errors because texify --quiet switches the log to file:line: styleOpen
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
SublimeText/LaTeXTools#1718 ·