8
$\begingroup$

What are the disadvantages of using elastic net in comparison to lasso. I know that the elastic net is able to select groups of variables when they are highly correlated.

  1. It doesn't have the problem of selecting more than $n$ predictors when $p \gg n$. Whereas lasso saturates when $p \gg n$.

  2. When there are highly correlated predictors lasso tends to just pick one predictor out of the group.

  3. When $n \gg p$ and the predictors are correlated, the prediction performance of lasso is smaller than that of ridge.

All these disadvantages of lasso are overcome by the elasic net.

What I don't understand is when should the lasso be then used? Is there any reason to use it when the elastic net performs better than lasso? What are the disadvantages of using elastic net in some cases? In which cases would the lasso be a better choice?

$\endgroup$
9
$\begingroup$

One disadvantage is the computational cost. You need to cross-validate the relative weight of L1 vs. L2 penalty, $\alpha$, and that increases the computational cost by the number of values in the $\alpha$ grid.

Another disadvantage (but at the same time an advantage) is the flexibility of the estimator. With greater flexibility comes increased probability of overfitting. It may be that the optimal $\alpha$ for the population and for the given sample size is $0$, turning elastic net into lasso, but you happen to choose a different value due to chance (because that value delivers better performance when cross-validating in the particular sample).

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Minor comment with regards to your second point: lasso and elastic net are estimators for the same model. As such, there is no difference in flexibility of the model. $\endgroup$ – hejseb May 9 '18 at 22:15
  • 2
    $\begingroup$ @hejseb: With LASSO, there is a single parameter that is optimized over during cross-validation: $\lambda$. In elastic-net, you can optimize over both $\alpha$ and $\lambda$, meaning more opportunity for overfitting during the cross-validation selection process. On the other hand, just using the default values of $\alpha$ tend to perform really well, so often only $\lambda$ is optimized over. So I disagree with your statement, because I consider $\alpha$ and $\lambda$ a part of the model (although I understand the ambiguity). $\endgroup$ – Cliff AB May 9 '18 at 22:33
  • 3
    $\begingroup$ @hejseb, excellent point! Now corrected. Cliff AB, I think the model definition need not include the tuning parameters of the estimator (lasso, elastic net, ...) – which $\alpha$ and $\lambda$ are – so to me hejseb's comment makes a lot of sense. I think along the lines of defining a model for the population (a linear model in this case) and estimating its parameters (which include $\beta$s but not $\alpha$ or $\lambda$) by some estimator. $\endgroup$ – Richard Hardy May 10 '18 at 7:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.