Hacktoberfest 2026: những issue maintainer đã đánh dấu cho tháng Mười, đang mở và phù hợp người mới. Xem issue Hacktoberfest

Feature (Phase 2): Implement named ingestion sources (RSS feeds, GitHub markdown/YAML lists, Apify actors) & deadline expiry rules

Đang mở
  • Có gì không ổn?
#83 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức phù hợp với người mới
18/100
Loại issue
Tính năng
Độ rõ ràng
Khá rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
github, typescript
Lĩnh vực
backend, data-engineering

Hướng nghiên cứu

Start with src/lib/ingestion/sources/index.ts, which exports SOURCES = [], then read src/lib/ingestion/run-ingestion.ts and src/app/api/cron/link-health/route.ts. This is a broad tracking issue spanning many adapters, so a newcomer should pick one slice, such as the past-deadline drop in run-ingestion.ts, and confirm scope with maintainers first. Done means the slice works and has unit tests.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

enhancement

Description

Tracked from docs/2026-09-18-improvement-plan.md (§4 Phase 2) and docs/2026-09-18-github-resource-repos.md. Complements #40 with the concrete source adapters, parsers, and deadline-expiry rules from the Sept 18 research sweep.

Currently, the ingestion pipeline scaffolding (src/lib/ingestion/, /api/cron/ingest-resources, /admin/ingestion) is built, but src/lib/ingestion/sources/index.ts exports SOURCES = [].

Concrete Sources to Implement (src/lib/ingestion/sources/)

1. Free RSS & JSON Feeds (plain fetch)
  • Opportunity Desk (https://opportunitydesk.org/feed/): scholarships, fellowships, grants, contests; extract deadline from description text via regex.
  • Scholars4Dev (https://www.scholars4dev.com/feed/): international scholarships with deadline data.
  • Brabble.ai JSON/RSS (https://brabble.ai/): India-centric hackathons, competitions, and coding contests with structured closing dates (with attribution).
2. GitHub Resource Lists (docs/2026-09-18-github-resource-repos.md)

Build two generic parsers (Markdown Table parser and Markdown Link List / YAML parser) with GitHub Commits API change-detection:

  • yashisrani/List-of-OpenSource-Programs (YAML/Markdown with stipends & timelines)
  • ayush-sleeping/Every-Open-Source-Programs (month-by-month tables)
  • deepanshu1422/List-Of-Open-Source-Internships-Programs
  • sdhutchins/awesome-edu-perks (tag US/Global region)
  • Official GitHub Student Developer Pack partner page (https://education.github.com/pack)
3. Apify Actor Sources (APIFY_TOKEN via run-sync-get-dataset-items)
  • saadithya/scholarships-competitions-internships-extractor: normalized YYYY-MM-DD deadlines across Scholarships, Fellowships, Internships.
  • gabrielaxy/hackathon-aggregator: Devpost, Devfolio, ETHGlobal, DoraHacks start/end dates (kind = 'program').
  • solidcode/unstop-scraper / dami_studio/unstop-scraper: filter strictly to scholarship + hackathon + named flagship internships (respecting the "not a job board" non-goal).
  • blackfalcondata/internshala-scraper: filter strictly to curated flagship programs (InStep, TCS Research, Microsoft, Amazon), not the firehose.

Pipeline & Link-Health Rules

  1. Drop past deadlines at fetch time: In run-ingestion.ts, any candidate with deadline < now() is dropped before inserting into ingestion_candidates.
  2. Link-health deadline expiry (src/app/api/cron/link-health/route.ts):
    • If a live resource has deadline < now() and no recurrence, automatically flip status = 'expired'.
    • If a live resource has deadline < now() with recurrence, clear deadline = null and keep its "reopens ~month" recurrence label active.

Acceptance Criteria

  • SOURCES in src/lib/ingestion/sources/index.ts includes working adapters for RSS feeds, GitHub markdown/YAML lists, and Apify actors (when APIFY_TOKEN is set).
  • Candidates with deadline < now() are dropped during ingestion.
  • Link-health cron expires non-recurring past-deadline resources and resets recurring past-deadline resources.
  • Unit tests cover the markdown table parser, RSS deadline extractor, past-deadline filtering, and recurrence rollover.

Files Involved

  • src/lib/ingestion/sources/index.ts
  • src/lib/ingestion/sources/*.ts (new adapters)
  • src/lib/ingestion/run-ingestion.ts
  • src/app/api/cron/link-health/route.ts
  • vercel.json
Ngôn ngữ chính
TypeScript
Star
1
Fork
6
Merge trung bình
23 giờ 53 phút
Pull request đã merge (30 ngày)
27

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của Your-Voldemort/StudentStack

Tất cả issue của Your-Voldemort/StudentStack

Issue tương tự

Thêm issue về TypeScript

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.