Support frequency weights on context rows for binomial / aggregated count data

Aperta
  • Qualcosa non va?
#1,318 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
42/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python
Ambito
machine-learning

Direzione di ricerca

Start by tracing fit/predict, the column inducing blocks and ICLBlock attention, CellEmbedding imputation, and the recipe's context-fitted transforms. The issue identifies these components but names no files or tests, so locate their implementations and existing test coverage first. Done means context-row weights are supported and integer weights match explicit row repetition, including weighted preprocessing.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Binomial problems are common: each group has features x, n trials and k successes, and the goal is to predict the success probability p(x). Examples include conversion rates, defect rates, dose–response and clinical trial arms.

Today, the only exact way to use KumoTabular for this is to expand every trial into its own context row (k rows with label 1 and n − k rows with label 0). That makes the context sum(n) rows instead of about 2 × groups. The ICL attention cost grows with context length, and the context-row limit is reached much sooner. Workarounds that regress on k/n or the empirical logit lose the trial counts. They also do badly when p is close to 0 or 1, where most groups have k = n.

It would help a lot if fit/predict accepted a sample_weight for context rows. A row with weight m should behave exactly like m identical copies of that row. A binomial group then becomes at most two weighted rows.

Add an optional per-context-row weight w. KumoTabular has no positional encoding over rows, so identical copies stay identical through the network. Repeating a row m times is therefore exactly equivalent to:

  1. Attention over context rows: in the column inducing blocks and the ICLBlock layers, add log(w) to the attention scores of each context row. Pass sum(w) as key_len to LogScale. No other attention block needs changes. This mostly means letting SDPA accept a float attn_mask.
  2. Missing-value imputation: use a weighted mean in CellEmbedding.
  3. Preprocessing: fit the recipe's context-fitted transforms with weights. These are the scalers, power transforms, category counts and target standardization.

A weight of 0 removes a row. Integer weights reproduce explicit repetition up to float rounding.

I prototyped this on the small classifier, and changes 1 and 2 above are enough for the network to match sdm run on explicitly repeated rows (max |Δ| ~1e-5). On a synthetic binomial benchmark (180 groups, 1448 trials) the context shrank from 1448 to 188–299 rows. Inference was 7–9× faster with identical predicted probabilities (|Δp| ≤ 2e-6).

Lingua principale
Python
Stelle
429
Fork
46
Merge medio
1g 3h
PR unite (30g)
231

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di NVIDIA/structured-data-models

Tutte le issue di NVIDIA/structured-data-models

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.