0
$\begingroup$

I have a question regarding the interpretation of the trace of coefficients when running Elastic net with the package glmnet in R.

This is the plot I obtain with alpha = 0.5

Elastic net trace of variable coefficients, alpha = 0.5

My understanding is that the numbers on the top are the number of non-zero coefficients included in the model at the given value of lambda (and alpha).

I have trouble understanding why the number of non-zero coefficients suddenly starts increasing with lambda? I would expect more variable coefficients be set to zero as lambda increases, from looking at the formula for the elastic net penalty.

I am hoping someone can explain if the pattern I see is due to an error that I made, me misinterpreting the graph, or something that can be rationally explained.

I used the following code:

fit.elnet <- glmnet(x.train, y.train, family="gaussian", alpha=.5)
plot(fit.elnet, xvar="lambda")

Where y.train contains only the time series of prices i am trying to explain and x.train contains only timeseries of explanatory variables. The data has high multicollinearity, if that is in any way useful.

EDIT:

These show the same plot but with alpha = 0.1 and alpha = 0.9 respectively. enter image description here

$\endgroup$
  • $\begingroup$ Since the elastic-net penalty is $\lambda( (1-\alpha)\| \beta \|^{2}_{2}/2 + \alpha \| \beta \|_{1})$, it's possible that what you are seeing is the penalty being dominated by $\| \beta \|_1$ for smaller $\lambda$, and the reverse for larger $\lambda$ due to squares of the $\ell_2$ penalty. You could confirm or deny this by comparing the plots for various $\alpha$ values. $\endgroup$ – Nutle May 8 at 9:28
  • $\begingroup$ Hi! Thank for your quick response. This was something i considered, but i couldnt figure out how to confirm it. I have edited my post to contain two additional graphs, one for alpha = 0.1 and alpha = 0.9. What am i looking for? $\endgroup$ – Kris May 8 at 10:42
  • $\begingroup$ One possible way to check how do both penalties look like at each $\lambda$ value: require(magrittr);(coefficients(fit.elnet) %>% apply(., 2, function(x){ c(sum(abs(x)), sum( x^2 )/2 ) }))*fit.elnet$lambda $\endgroup$ – Nutle May 8 at 11:19
  • $\begingroup$ Running this code perfectly illustrates the first explanation you made. Thank you! $\endgroup$ – Kris May 8 at 11:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.