research: back-test a second-gift conversion model on DonorsChoose

Aperta
  • Qualcosa non va?
#346 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
28/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python
Ambito
data, machine-learning

Direzione di ricerca

Start by getting the local DonorsChoose file as described in docs/results/datasets.md, then read philanthropy.datasets.load_donorschoose to see which first-gift fields are available. Use philanthropy.model_selection.compare_to_rule for the rule baselines and paired bootstrap, and scripts/make_results_pages.py for the Results row format. Done means a test-year report of top-10% hit rate and AUC against the best validation rule, recorded as either a Results row or a rejected note with the numbers.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

enhancement help wanted roadmap
Goal

Find out whether a model beats a simple rule at picking which first-time donors will give again within 12 months.

Acceptance criteria
  • Population: donors whose first gift falls in fiscal year T. Label: another gift within 12 months of the first. load_donorschoose truncates gift_date to the first of the month, so define "within 12 months" in whole months and say so; a second gift in the same month as the first is ambiguous and must be handled explicitly (drop or count, stated).
  • Baseline rules: "first gift above the median", plus one-column rules on what the loader actually returns (donor_type, was_matched, the two gift-card flags). There is no recurring-gift field in this file. The reported baseline is the best rule on the validation fold.
  • Model: DonorPropensityModel on features known at the first gift only (amount, month, donor_type, was_matched, gift-card flags), so nothing after the first gift leaks in. load_donorschoose returns no project fields; joining them would need a second file and a note on its licence.
  • Protocol: choose settings on a validation fiscal year, report once on a later test year: top-10% hit rate and AUC with a paired bootstrap interval against the best rule.
  • Outcome recorded either way: a Results row if it wins, or a "rejected" note with the numbers if it doesn't.
Scope

Research and a Results-page row. No new public estimator unless the model clearly wins on the test year.

Where to start
  • philanthropy.datasets.load_donorschoose (local file; docs/results/datasets.md says where to get it)
  • philanthropy.model_selection.compare_to_rule for the bootstrap and verdict logic to copy. It only accepts task in {"upgrade", "lapse", "ask"}, so it cannot run this comparison as is; adding a task is part of the public API and only follows an adopt result
  • scripts/make_results_pages.py for how existing rows are generated
Dependencies
  • None

Phase order is a release plan, not a rule against picking up independent work early. Comment on this issue before you start so two people don't build the same thing.

Lingua principale
Python
Stelle
11
Fork
23
Merge medio
1g 11h
PR unite (30g)
30

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di PhilanthroPy-Project/PhilanthroPy

Tutte le issue di PhilanthroPy-Project/PhilanthroPy

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.