I have a question regarding the interpretation of the trace of coefficients when running Elastic net with the package glmnet in R.

This is the plot I obtain with alpha = 0.5

Elastic net trace of variable coefficients, alpha = 0.5

My understanding is that the numbers on the top are the number of non-zero coefficients included in the model at the given value of lambda (and alpha).

I have trouble understanding why the number of non-zero coefficients suddenly starts increasing with lambda? I would expect more variable coefficients be set to zero as lambda increases, from looking at the formula for the elastic net penalty.

I am hoping someone can explain if the pattern I see is due to an error that I made, me misinterpreting the graph, or something that can be rationally explained.

I used the following code:

fit.elnet <- glmnet(x.train, y.train, family="gaussian", alpha=.5)
plot(fit.elnet, xvar="lambda")

Where y.train contains only the time series of prices i am trying to explain and x.train contains only timeseries of explanatory variables. The data has high multicollinearity, if that is in any way useful.


These show the same plot but with alpha = 0.1 and alpha = 0.9 respectively. enter image description here

  • $\begingroup$ Since the elastic-net penalty is $\lambda( (1-\alpha)\| \beta \|^{2}_{2}/2 + \alpha \| \beta \|_{1})$, it's possible that what you are seeing is the penalty being dominated by $\| \beta \|_1$ for smaller $\lambda$, and the reverse for larger $\lambda$ due to squares of the $\ell_2$ penalty. You could confirm or deny this by comparing the plots for various $\alpha$ values. $\endgroup$ – Nutle May 8 at 9:28
  • $\begingroup$ Hi! Thank for your quick response. This was something i considered, but i couldnt figure out how to confirm it. I have edited my post to contain two additional graphs, one for alpha = 0.1 and alpha = 0.9. What am i looking for? $\endgroup$ – Kris May 8 at 10:42
  • $\begingroup$ One possible way to check how do both penalties look like at each $\lambda$ value: require(magrittr);(coefficients(fit.elnet) %>% apply(., 2, function(x){ c(sum(abs(x)), sum( x^2 )/2 ) }))*fit.elnet$lambda $\endgroup$ – Nutle May 8 at 11:19
  • $\begingroup$ Running this code perfectly illustrates the first explanation you made. Thank you! $\endgroup$ – Kris May 8 at 11:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.