Support frequency weights on context rows for binomial / aggregated count data

オープン
  • 何か問題がありますか？
#1,318 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
5/5
見積もり時間
1週間以上
初心者へのやさしさ
42/100
issue の種類
機能追加
明瞭さ
おおむね明確
活発さ
活発
技術スタック
python
領域
machine-learning

調査の方向性

Start by tracing fit/predict, the column inducing blocks and ICLBlock attention, CellEmbedding imputation, and the recipe's context-fitted transforms. The issue identifies these components but names no files or tests, so locate their implementations and existing test coverage first. Done means context-row weights are supported and integer weights match explicit row repetition, including weighted preprocessing.

索引モデルが issue の本文から書いたものです。

説明

Binomial problems are common: each group has features x, n trials and k successes, and the goal is to predict the success probability p(x). Examples include conversion rates, defect rates, dose–response and clinical trial arms.

Today, the only exact way to use KumoTabular for this is to expand every trial into its own context row (k rows with label 1 and n − k rows with label 0). That makes the context sum(n) rows instead of about 2 × groups. The ICL attention cost grows with context length, and the context-row limit is reached much sooner. Workarounds that regress on k/n or the empirical logit lose the trial counts. They also do badly when p is close to 0 or 1, where most groups have k = n.

It would help a lot if fit/predict accepted a sample_weight for context rows. A row with weight m should behave exactly like m identical copies of that row. A binomial group then becomes at most two weighted rows.

Add an optional per-context-row weight w. KumoTabular has no positional encoding over rows, so identical copies stay identical through the network. Repeating a row m times is therefore exactly equivalent to:

  1. Attention over context rows: in the column inducing blocks and the ICLBlock layers, add log(w) to the attention scores of each context row. Pass sum(w) as key_len to LogScale. No other attention block needs changes. This mostly means letting SDPA accept a float attn_mask.
  2. Missing-value imputation: use a weighted mean in CellEmbedding.
  3. Preprocessing: fit the recipe's context-fitted transforms with weights. These are the scalers, power transforms, category counts and target standardization.

A weight of 0 removes a row. Integer weights reproduce explicit repetition up to float rounding.

I prototyped this on the small classifier, and changes 1 and 2 above are enough for the network to match sdm run on explicitly repeated rows (max |Δ| ~1e-5). On a synthetic binomial benchmark (180 groups, 1448 trials) the context shrank from 1448 to 188–299 rows. Inference was 7–9× faster with identical predicted probabilities (|Δp| ≤ 2e-6).

主要言語
Python
スター
429
フォーク
46
平均マージ
1日 6時間
マージ済み PR（30日）
227

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

NVIDIA/structured-data-models のほかの issue

NVIDIA/structured-data-models の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。