Handling ID columns in TabularPrediction

Many tabular datasets contain an "ID" integer column that is useless as a predictive feature. Such features are often actively harmful to include as predictors (eg. because they take larger values in test data than in training data).

Currently, AutoGluon users should manually declare such columns via the keyword-arg id_columns in TabularPrediciton.fit() and then AutoGluon will ignore them.

However most users are unlikely to do this, expecting AutoML to automatically handle it. Once feature-pruning is added into AG, it may be able to resolve this issue, but only given sufficient time_limits (and AG will still waste time training models with the ID feature included at first).

Proposal: An ID-specific filter to automatically detect such features and remove them before training begins. This filter can be based on: the column-name (e.g. is it ["id", "index"]), the column-index (is it the first column), as well as the column-values (are they integers & possibly contiguous). The filter can also check if this column contains any marginal predictive signal on its own (eg. via a quick k-nearest-neighbors fit on a data subsample using solely this column as a feature).

Pros of such a filter:

• can improve accuracy when there is insufficient time_limits for feature-pruning

• avoids wasting time training models with obviously-bad feature included

Cons of such a filter:

• May have false positives. In the worst-case, imagine a dataset where label = 1 if ID_feature > 100 & other_feature > 1, = 0 otherwise; here dropping ID_feature erroneously will really harm accuracy (but such datasets are presumably rare).

• AG behavior will exhibit discontinuity where tiny change in such a column (eg. changing its name by a little or just one value) can cause big change in AG behavior. This might be addressed via a clear output message like:

“Column ‘id’ is ignored by autogluon. If this column should be used as a predictive feature, please rename it to something besides: ['id', 'index', etc…]”

We're eager to hear community input/ideas regarding this proposal!