Different alpha selection strategies in LinearModelCV
#6 630 ouverte le 6 avr. 2016
Métriques du dépôt
- Stars
- (66 084 stars)
- Métriques de merge PR
- (Merge moyen 10j) (90 PRs mergées en 30 j)
Description
LinearModelCV selects the alpha with the minimum MSE across folds:
for l1_ratio, l1_alphas, mse_alphas in zip(l1_ratios, alphas,
mean_mse):
i_best_alpha = np.argmin(mse_alphas)
This is not always advisable, ie we might instead want to trade a slightly "worse" alpha for a less complex model (less variables), or even a more complex one (in my case for example I'd be happy to trade a higher MSE for a few more variables selected).
Glmnet has the option to use either the min-MSE alpha, or a less complex one within 1 standard deviation of the minimum (they call it lambda.1se).
We could add an argument taking three values: "alpha.min", "alpha.1se+", "alpha.1se-" or something similar, to indicate three possible alpha selections:
- minimum, like now
- least complex within 1 se of min
- most complex within 1 se of min
Or even use an argument whose default value is the function np.argmin, so that the user can specify his own alpha-selection function (still limited to only mse_alphas, though).
What do you think?