GratefulPatientFeaturizer/EncounterTransformer: index-encounter keying vs. the PHI-free bundle contract need a design decision

Abierto
  • ¿Algo no está bien?
#110 1 comentario 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
25/100
Tipo de issue
Refactorización
Claridad
Bien especificado
Estado de actividad
Tranquilo
Stack tecnológico
python, scikit-learn
Área
data

Línea de trabajo

Read philanthropy/preprocessing/_grateful_patient.py, especially merge_key, last_discharge, and __getstate__, then review the PHI-free bundle contract in docs/explanation/compliance_considerations.md and the issue’s linked context in #78. The issue is closed by a written decision on the tradeoff; if retaining latest-discharge keying, document the recurrent-patient limitation in the compliance considerations file.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

help wanted question roadmap
Status (2026-11-04)

Still undecided, and neither half of the 2026-09-06 proposal below has landed. Checked against e410de6: GratefulPatientFeaturizer.fit still keeps one last_discharge per donor (grouped[discharge_col].max() after the as_of cutoff), and no doc mentions the repeat-patient effect.

  • The "document it now" half is filed as #361 (docs only, no decision needed).
  • The per-donor sorted discharge-date array (option 2 as refined in that comment) is not filed. If it is accepted, it becomes its own feature issue with the searchsorted design, and this issue closes with the decision.
  • Relation to 1.0 (#351): GratefulPatientFeaturizer, EncounterTransformer and EncounterRecencyTransformer are all Tier 2 in docs/reference/index.md, which may change in a minor release with a Breaking note. So per-gift keying can ship in a 1.x minor; what 1.0 needs is the limitation documented (#361), not the redesign.

The one decision left for a person: adopt the sorted-date-array design as the plan (and file it), or keep latest-discharge keying permanently.

The conflict

GratefulPatientFeaturizer and EncounterTransformer key on the donor's
latest discharge (grep -n "merge_key" philanthropy/preprocessing/_grateful_patient.py,
groupby on merge_key then take last_discharge). For a donor with more than
one encounter, a gift is measured against whichever discharge happens to be
most recent at fit/transform time, not the discharge that was actually closest
in time to that specific gift. Recurrent patients (oncology, transplant) are
the people this most misrepresents.

The fix needs the raw per-gift, per-encounter rows available at transform
time so each gift can be joined to its own nearest prior discharge. That is
exactly what __getstate__ is designed to strip
(grep -n "__getstate__" -A8 philanthropy/preprocessing/_grateful_patient.py):
raw encounter_df is PHI-bearing and is replaced with None on serialisation
so a saved model bundle never carries patient rows (#78).

Per-gift keying and the PHI-free bundle contract are in direct conflict. This
is not a bug with an obvious patch; it needs a design decision on which
guarantee gives way, or a third option (e.g. storing only a de-identified
per-donor encounter-to-gift offset at fit time, if that is precise enough).

Why it matters now

This is the one item from the 2026-08-13 audit that landed as "documented
limitation" rather than "fixed" going into the JOSS submission. It affects the
grateful-patient timing story the paper's Software design section describes as
solved via as_of; per-gift keying is a separate, still-open gap in the same
story.

What would close this

A written decision (this issue, a discussion, or a design doc) on one of:

  1. Keep latest-discharge keying, document the recurrent-patient
    misattribution explicitly as an accepted limitation in
    docs/explanation/compliance_considerations.md.
  2. Store a de-identified per-donor "days since nearest prior discharge, as of
    each gift date" summary at fit time (no raw dates, no diagnosis), so
    __getstate__ still has nothing PHI-bearing to strip.
  3. Some other resolution.

This needs a person to decide the tradeoff; it cannot be closed by a PR alone.

Lenguaje dominante
Python
Estrellas
11
Forks
23
Merge medio
2 d 19 h
PR fusionados (30 d)
31

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de PhilanthroPy-Project/PhilanthroPy

Todos los issues de PhilanthroPy-Project/PhilanthroPy

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.