Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Feature (Phase 2): Implement named ingestion sources (RSS feeds, GitHub markdown/YAML lists, Apify actors) & deadline expiry rules

Aperta
  • Qualcosa non va?
#83 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
18/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
github, typescript
Ambito
backend, data-engineering

Direzione di ricerca

Start with src/lib/ingestion/sources/index.ts, which exports SOURCES = [], then read src/lib/ingestion/run-ingestion.ts and src/app/api/cron/link-health/route.ts. This is a broad tracking issue spanning many adapters, so a newcomer should pick one slice, such as the past-deadline drop in run-ingestion.ts, and confirm scope with maintainers first. Done means the slice works and has unit tests.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

enhancement

Description

Tracked from docs/2026-09-18-improvement-plan.md (§4 Phase 2) and docs/2026-09-18-github-resource-repos.md. Complements #40 with the concrete source adapters, parsers, and deadline-expiry rules from the Sept 18 research sweep.

Currently, the ingestion pipeline scaffolding (src/lib/ingestion/, /api/cron/ingest-resources, /admin/ingestion) is built, but src/lib/ingestion/sources/index.ts exports SOURCES = [].

Concrete Sources to Implement (src/lib/ingestion/sources/)

1. Free RSS & JSON Feeds (plain fetch)
  • Opportunity Desk (https://opportunitydesk.org/feed/): scholarships, fellowships, grants, contests; extract deadline from description text via regex.
  • Scholars4Dev (https://www.scholars4dev.com/feed/): international scholarships with deadline data.
  • Brabble.ai JSON/RSS (https://brabble.ai/): India-centric hackathons, competitions, and coding contests with structured closing dates (with attribution).
2. GitHub Resource Lists (docs/2026-09-18-github-resource-repos.md)

Build two generic parsers (Markdown Table parser and Markdown Link List / YAML parser) with GitHub Commits API change-detection:

  • yashisrani/List-of-OpenSource-Programs (YAML/Markdown with stipends & timelines)
  • ayush-sleeping/Every-Open-Source-Programs (month-by-month tables)
  • deepanshu1422/List-Of-Open-Source-Internships-Programs
  • sdhutchins/awesome-edu-perks (tag US/Global region)
  • Official GitHub Student Developer Pack partner page (https://education.github.com/pack)
3. Apify Actor Sources (APIFY_TOKEN via run-sync-get-dataset-items)
  • saadithya/scholarships-competitions-internships-extractor: normalized YYYY-MM-DD deadlines across Scholarships, Fellowships, Internships.
  • gabrielaxy/hackathon-aggregator: Devpost, Devfolio, ETHGlobal, DoraHacks start/end dates (kind = 'program').
  • solidcode/unstop-scraper / dami_studio/unstop-scraper: filter strictly to scholarship + hackathon + named flagship internships (respecting the "not a job board" non-goal).
  • blackfalcondata/internshala-scraper: filter strictly to curated flagship programs (InStep, TCS Research, Microsoft, Amazon), not the firehose.

Pipeline & Link-Health Rules

  1. Drop past deadlines at fetch time: In run-ingestion.ts, any candidate with deadline < now() is dropped before inserting into ingestion_candidates.
  2. Link-health deadline expiry (src/app/api/cron/link-health/route.ts):
    • If a live resource has deadline < now() and no recurrence, automatically flip status = 'expired'.
    • If a live resource has deadline < now() with recurrence, clear deadline = null and keep its "reopens ~month" recurrence label active.

Acceptance Criteria

  • SOURCES in src/lib/ingestion/sources/index.ts includes working adapters for RSS feeds, GitHub markdown/YAML lists, and Apify actors (when APIFY_TOKEN is set).
  • Candidates with deadline < now() are dropped during ingestion.
  • Link-health cron expires non-recurring past-deadline resources and resets recurring past-deadline resources.
  • Unit tests cover the markdown table parser, RSS deadline extractor, past-deadline filtering, and recurrence rollover.

Files Involved

  • src/lib/ingestion/sources/index.ts
  • src/lib/ingestion/sources/*.ts (new adapters)
  • src/lib/ingestion/run-ingestion.ts
  • src/app/api/cron/link-health/route.ts
  • vercel.json
Lingua principale
TypeScript
Stelle
1
Fork
6
Merge medio
23h 53m
PR unite (30g)
27

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di Your-Voldemort/StudentStack

Tutte le issue di Your-Voldemort/StudentStack

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.