[suggestion] Awareness — local-first public-text ingestion engine
Nobody has claimed this yet.
Assessment
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Newbie friendliness
- 74/100
- Issue type
- Documentation
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- fastapi, python
- Domain
- content, documentation
Research direction
Start by reading CONTRIBUTING and locating projects.yaml in the repository. Add the Awareness project using the details and repository URL from this issue, then open a pull request; done means the entry follows the file's existing format and includes the requested application information.
Written by the indexing model from the issue text.
Description
Project name
Awareness
Repo URL
https://github.com/nazmiefearmutcu/awareness
Description (description-only, per CONTRIBUTING)
Local-first public-text ingestion engine. Backfills any historical date range from Common Crawl + FineWeb and live-tails RSS / sitemaps / GDELT into Apache Iceberg, queryable from DuckDB. Single process, no cluster required.
Suggested tags
internet, dev.storage (data-pipeline / storage)
Why it fits
- Pure Python (FastAPI + pyiceberg + duckdb + httpx)
- Self-contained application (web UI ships with the FastAPI server at
/) - Long-running (live tail) + one-shot (backfill) modes
- Not a library / not a notebook / not a tutorial
Status
- v0.1.0 released, MIT licensed
- README has a built-in 'Research Workbench' UI section with 4 screenshots
- Architecture diagram in README
Happy to add to projects.yaml directly via PR if preferred — let me know.
- Dominant language
- Jupyter Notebook
- Stars
- 18k
- Forks
- 2.7k
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from mahmoud/awesome-python-applications
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Add kaydet Open
Difficulty 1/5 Under an hour Newbie friendliness 68/100
-
Difficulty 1/5 Under an hour Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
All issues in mahmoud/awesome-python-applications
Similar issues
-
community first-timers-only good first issue hacktoberfest help wanted low hanging fruit up-for-grabs
Difficulty 1/5 Under an hour Newbie friendliness 95/100
-
Ecosystem: ClawMetry — the Qwen Code reader is now free and open source (follow-up to #9294 / #9338) Opencategory/integration priority/P3 scope/documentation status/ready-for-human type/feature-request
Difficulty 1/5 Under an hour Newbie friendliness 84/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
TheOdinProject/curriculum#31408 ·
-
archived coursework help wanted scope: curriculum
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
freeCodeCamp/freeCodeCamp#70260 · 1 comment ·
-
雪鸿集 Open待处理 申请收录
Difficulty 1/5 Under an hour Newbie friendliness 72/100
travellings-link/travellings#4000 · 1 comment ·