# elastic net min deviance when lambda tends to 0

I'm trying to use glmnet packaget to perform elastic net model for binomial regression. I got the following graph:

What I see from this is that increasing the lambda never decreases the deviance, so does this mean that no penalization is better that any penalization over $L_1$ & $L_2$?

I'm not sure why you say that the deviance never decreases; it is certainly decreasing at large values of $\log(\lambda)$ on the rightmost edge of the plot to the left. However, the deviance is relatively flat over a large range of values of $\log(\lambda)$. Yet shrinkage is being applied, as the values on the upper x-axis (side = 3) indicate: the number of variables in the model declines consistently and markedly over the region of no or little change in deviance.
What this shows is that for models over this region, up to about $\log(\lambda) = -6$, shrinkage is being applied without affecting the CV fit of the model as assessed via CV binomial deviance. This is good, as it indicates that the elastic net is doing what you wanted it to, to perform feature selection whilst handling correlated covariates.
• @GabyP You are mixing up $\alpha$ and $\lambda$. The former, $\alpha$ is the mixing of L1 and L2 penalties, and is set as argument alpha in the cv.glmnet call. $\lambda$ is the amount of shrinkage applied. cv.glmnet optimises over $\lambda$ only. You need to do your own procedure if you wish to tune over both $\lambda$ and $\alpha$. If this is what you were trying to do, consider the caret package, which should be able to tune over both parameters for you using glmnet models. Oct 29, 2014 at 3:05
• The penalization is in terms of shrinkage of model coefficients. On the left you have the CV binomial deviance for the full, unpenalized model; on the right you have a heavily shrunk fit with large penalties. What this plot is showing you is that you can shrink the coefficients of a lot covariates by an amount sufficient to remove them from the model (their coefficient becomes zero) without affecting the model fit - the deviance of the model doesn't change much. Only at large penalties (large values of $\log(\lambda)$) does the model start to loose fit relative to the full model. Oct 29, 2014 at 3:12