Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Explainers: Monitoring Fairness in Production, Label Shift and Fairness, and Delayed Feedback

Chiusa
  • Qualcosa non va?
#1,100 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
35/100
Tipo di issue
Documentazione
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
jupyter, python, scikit-learn
Ambito
documentation, machine-learning

Direzione di ricerca

Read CONTRIBUTING.md, CLAUDE.md, and a close sibling explainer under explainers/; inspect faircode.benchmark.load_audit and the repository audits first. Add three explainers with runnable analyses, verify the cited sources, update the listed documentation and data files, then run make build-explainers and make lint. Done means all three explainers and required generated files are included and each analysis runs from the repository root.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

documentation help wanted

The repo has Model Drift, Distribution Shift and Feedback Loop Bias, but nothing on the practice of watching a deployed model's fairness, or on two specific ways measurement goes wrong after deployment. Add all three in one pull request.

explainers/monitoring-fairness-in-production.md - How Do You Monitor Fairness After Deployment?

Track a fairness metric over time with an interval and a control-chart rule (a rolling window, an alert when the interval excludes the launch value). Simulate it on a real audit by splitting the data into time-like batches (or by a column with a natural order), injecting a drift in one group, and showing how many batches pass before the alert fires for different batch sizes. Cover window size, minimum group size and alert fatigue.

Key citation to verify: Cite a primary source on drift detection such as Gama et al. (2014), "A Survey on Concept Drift Adaptation" (ACM Computing Surveys) - confirm.

explainers/label-shift-and-fairness.md - What Happens to Fairness Under Label Shift?

The base rate of the outcome changes between training and deployment while the feature-given-label relationship stays put. Show on a real audit, by resampling the test set to different base rates for one group, how demographic parity, equal opportunity and predictive parity respond differently, and why a model fair at launch can fail on predictive parity alone.

Key citation to verify: Lipton, Wang, Smola (2018), "Detecting and Correcting for Label Shift with Black Box Predictors" (ICML) - confirm.

explainers/delayed-feedback.md - What Is Delayed and Censored Feedback?

Outcomes arrive late, or never, for some people, so recent data under-counts positives and the groups with slower outcomes look better than they are. Simulate it on a real audit by hiding the outcomes of the most recent rows with a delay that differs by group, and measure the bias it adds to a monitored gap. Contrast with selective labels (survivorship) and with feedback loops.

Key citation to verify: Chapelle (2014), "Modeling Delayed Feedback in Display Advertising" (KDD) and Lakkaraju et al. (2017), "The Selective Labels Problem" (KDD) - confirm.

Related explainers: model-drift, distribution-shift, feedback-loop-bias, survivorship-bias, predictive-parity, disaggregated-evaluation.

See CONTRIBUTING.md: add explainers/<slug>.md plus a one-line entry in assets/explainers-data.json, then run make build-explainers; the build script generates the pages, sitemap, llms-full.txt and OG images, and llms.txt and the README table must list each explainer (scripts/check_explainer_count.py checks both). Match the structure of a close sibling explainer, use real code run against one of the repo's audits (faircode.benchmark.load_audit loads an audit exactly like the benchmark harness) and real, not invented, results, and if you quote a Fair Code result say whether it comes from results/ (current) or paper/results-frozen/ (the earlier reference snapshot), per CLAUDE.md. Verify every citation yourself before using it.

Done when: the 3 explainers, their README, CONTRIBUTING, ROADMAP, llms.txt and CHANGELOG entries, the website JSON entries and the regenerated generated files are merged, make lint passes, and each explainer's code is runnable from the repo root.

Lingua principale
HTML
Stelle
47
Fork
45
Merge medio
3h 2m
PR unite (30g)
85

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di yakew7/Fair-Code

Tutte le issue di yakew7/Fair-Code

Issue simili

Altre issue su Documentation

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.