1
$\begingroup$

I have been following a tutorial of LASSO regression with R commands (here), but I am confused at some point which I think is more conceptual.

I expected to get an estimate of the regression parameters, as well as a subset of the parameters that are meaningful to explain the response (sort of what would be obtained by using a backward stepwise regression). Instead, it seems that the output from this are the estimates of these coefficients at different values of a penalizing factor (λ).

So I am just guessing here but, is it possible to take the best value of λ (e.g. that with minimum AIC) to cut off the model on that value and get the corresponding estimates of these parameters? If so, how to do it?

In the following reproducible example... would the output mean that no factor is meaningful to predict mpg in the dataset of mtcars??

library("dplyr")
library("glmnet")
set.seed(123)
y <- mtcars %>% select(mpg) %>% scale(center = TRUE, scale = FALSE) %>% as.matrix()
X <- mtcars %>% select(-mpg) %>% as.matrix()
lambdas_to_try <- 10^seq(-3, 5, length.out = 100)
lasso_cv <- cv.glmnet(X, y, alpha = 1, lambda = lambdas_to_try,
                  standardize = TRUE, nfolds = 10)
plot(lasso_cv)
lambda_cv <- lasso_cv$lambda.min

model_cv <- glmnet(X, y, alpha = 1, lambda = lambda_cv, standardize = TRUE)

res <- glmnet(X, y, alpha = 1, lambda = lambdas_to_try, standardize = FALSE)
plot(res, xvar = "lambda")
legend("bottomright", lwd = 1, col = 1:6, legend = colnames(X), cex = .7)
abline(v = log(lambda_cv))
$\endgroup$
3
  • $\begingroup$ I'm having trouble finding a conceptual question here: your software outputs parameter estimates for each $\lambda$ in its search space and it also outputs statistics to identify the "best" (in several different ways), so what's left to do or understand? The "how to do it" question is very well explained and illustrated in the documentation for glmnet. $\endgroup$
    – whuber
    Feb 18, 2020 at 18:21
  • $\begingroup$ So as I understand, the model_cv object above is the regression fit using the best λ, but from there on I am kind of lost. If I read the content of model_cv$beta, I see in the example only three coefficients: cyl, hp and wt. But if I look to the plot, at the point where λ is minimum, it does not look like the values coefficients and the coefficients themselves are the same as the ones model_cv$beta. So summarizing, I do not feel confident or have not clear how to extract these coefficients because of these inconsistencies. $\endgroup$ Feb 19, 2020 at 9:10
  • $\begingroup$ You need to use cv.glmnet. $\endgroup$
    – whuber
    Feb 19, 2020 at 13:34

0

Browse other questions tagged or ask your own question.