1
$\begingroup$

I know Lasso eventually set some parameters to zero, acting like variable selection. I also read from paper talking about automated variable selection method like Stepwise AIC can be troublesome. So what are the advantages of using Lasso for variable selection over using the automated procedures like Stepwise AIC?

$\endgroup$
  • 2
    $\begingroup$ ESL and Introduction to Statistical Learning both summarize regularization - and are freely available. $\endgroup$ – charles Aug 22 '14 at 23:11
  • 2
    $\begingroup$ A lot has been written about why stepwise regression is a poor approach. If you believe that (and you should), then lasso would be "better" simply because it is less bad. $\endgroup$ – shadowtalker Aug 23 '14 at 19:04
2
$\begingroup$

Selection of variables on the basis of the magnitude of their observed regression coefficients, or on the basis of their statistical "significance" (and AIC is a 1-1 function of the P-value) will result in selection of variables because their effects were measured with error in the direction of being too far from zero. The lasso as well as other penalized methods penalize you for the context of how hard you have to work to find good predictors. Penalization lowers the chance of overstating a regression coefficient.

$\endgroup$
0
$\begingroup$

I am not 100% sure that this is right.

Stepwise AIC is designed to obtained a sparse estimator that works well on the training set (if you compute the MSE part of the AIC error on the training set)

The lasso where the penalty is chosen with cross validation is design to obtain the sparse linear model that minimize the prediction error

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.