Support frequency weights on context rows for binomial / aggregated count data

Abierto
  • ¿Algo no está bien?
#1,318 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
42/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
python
Área
machine-learning

Línea de trabajo

Start by tracing fit/predict, the column inducing blocks and ICLBlock attention, CellEmbedding imputation, and the recipe's context-fitted transforms. The issue identifies these components but names no files or tests, so locate their implementations and existing test coverage first. Done means context-row weights are supported and integer weights match explicit row repetition, including weighted preprocessing.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Binomial problems are common: each group has features x, n trials and k successes, and the goal is to predict the success probability p(x). Examples include conversion rates, defect rates, dose–response and clinical trial arms.

Today, the only exact way to use KumoTabular for this is to expand every trial into its own context row (k rows with label 1 and n − k rows with label 0). That makes the context sum(n) rows instead of about 2 × groups. The ICL attention cost grows with context length, and the context-row limit is reached much sooner. Workarounds that regress on k/n or the empirical logit lose the trial counts. They also do badly when p is close to 0 or 1, where most groups have k = n.

It would help a lot if fit/predict accepted a sample_weight for context rows. A row with weight m should behave exactly like m identical copies of that row. A binomial group then becomes at most two weighted rows.

Add an optional per-context-row weight w. KumoTabular has no positional encoding over rows, so identical copies stay identical through the network. Repeating a row m times is therefore exactly equivalent to:

  1. Attention over context rows: in the column inducing blocks and the ICLBlock layers, add log(w) to the attention scores of each context row. Pass sum(w) as key_len to LogScale. No other attention block needs changes. This mostly means letting SDPA accept a float attn_mask.
  2. Missing-value imputation: use a weighted mean in CellEmbedding.
  3. Preprocessing: fit the recipe's context-fitted transforms with weights. These are the scalers, power transforms, category counts and target standardization.

A weight of 0 removes a row. Integer weights reproduce explicit repetition up to float rounding.

I prototyped this on the small classifier, and changes 1 and 2 above are enough for the network to match sdm run on explicitly repeated rows (max |Δ| ~1e-5). On a synthetic binomial benchmark (180 groups, 1448 trials) the context shrank from 1448 to 188–299 rows. Inference was 7–9× faster with identical predicted probabilities (|Δp| ≤ 2e-6).

Lenguaje dominante
Python
Estrellas
429
Forks
46
Merge medio
1 d 6 h
PR fusionados (30 d)
227

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de NVIDIA/structured-data-models

Todos los issues de NVIDIA/structured-data-models

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.