Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Phase 4: Resource ingestion pipeline for automated content discovery

Aperta
  • Qualcosa non va?
#40 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
25/100
Tipo di issue
Funzionalità
Chiarezza
Da chiarire
Stato di attività
Attiva
Stack tecnologico
github, typescript
Ambito
backend, database, devops, full-stack

Direzione di ricerca

Start by reading src/lib/ingestion/run-ingestion.ts, src/lib/ingestion/sources/index.ts, and src/lib/ingestion/classify-match.ts, then trace the cron route and the admin actions. Review vercel.json and the existing ingestion flow before deciding how to divide the source adapters, scheduling, matching, monitoring, and review-workflow work. Done means the listed ingestion stages and priority sources operate through admin approval.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

enhancement

Description

The codebase already has a partial ingestion pipeline implementation in src/lib/ingestion/. This issue tracks completing it for automated content discovery from external sources.

Current State (from codebase exploration)

  • src/lib/ingestion/types.ts - TypeScript types for candidates
  • src/lib/ingestion/sources/index.ts - Source adapters (GitHub Student Pack, etc.)
  • src/lib/ingestion/classify-match.ts - Duplicate detection logic
  • src/lib/ingestion/run-ingestion.ts - Main ingestion orchestration
  • src/lib/ingestion/queries.ts - Database queries for candidates
  • src/app/api/cron/ingest-resources/route.ts - Cron endpoint
  • src/app/admin/(dashboard)/ingestion/page.tsx - Admin review UI
  • src/app/admin/(dashboard)/ingestion/actions.ts - Approve/reject actions

What's Needed to Complete

  1. Source implementations: Complete adapters for:
    • GitHub Student Pack (official API or scraping)
    • Major cloud providers (AWS, Azure, GCP student pages)
    • Popular tool vendor pages (JetBrains, Figma, Notion, etc.)
  2. Scheduling: Vercel Cron to run ingestion weekly
  3. Deduplication improvement: Better matching logic in classify-match.ts
  4. Admin review workflow: Already partially built, needs polish
  5. Source monitoring: Track which sources are healthy/stale

Files to Enhance

  • src/lib/ingestion/sources/ - add more source adapters
  • src/lib/ingestion/classify-match.ts - improve matching algorithm
  • src/app/api/cron/ingest-resources/route.ts - ensure cron works
  • vercel.json - add ingestion cron schedule

Ingestion Flow

  1. Cron triggers /api/cron/ingest-resources
  2. For each source: fetch raw data → normalize → classify (new vs possible duplicate)
  3. Store in ingestion_candidates table with matchType and matchedResourceId
  4. Admin reviews in /admin/ingestion → Approve (creates resource) or Reject
  5. Approved candidates → inserted into resources with approved = true

Priority Sources

  1. GitHub Student Pack (official partner list)
  2. AWS / Azure / GCP student pages
  3. Major IDE vendors (JetBrains, VS Code extensions)
  4. Design tools (Figma, Canva, Framer, Sketch)
  5. Productivity tools (Notion, Linear, Raycast, etc.)

Notes

  • This reduces manual admin work for keeping directory fresh
  • Complements community submissions (Phase 4)
  • Can run more frequently than link-health cron
Lingua principale
TypeScript
Stelle
1
Fork
6
Merge medio
11h 16m
PR unite (30g)
16

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di Your-Voldemort/StudentStack

Tutte le issue di Your-Voldemort/StudentStack

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.