I am using caret package to do L2-regularized multiple linear regression modeling. I am able to train the model and get the best tuned hyper parameter (lambda). The main goal however for me is to explain the most important features contributing to the performance of the model. I know that there is a function named, varImp, in caret package which list out important variables in order, but I have little bit difficulty in digesting the results. I have the following questions:

  1. Is there any similar method equivalent to glmnet::coef exist in caret to retrieve weights assigned to predictors?
  2. If I end up using caret::varImp function, what arguments I should supply to this in order to get the same results as I would get from glmnet::coef? Something on the line of (useModel =TRUE ?)

Hereby I am sharing the code that I am using train the model and get the variable importance using caret varImp function:

gridsearch_for_lambda =  data.frame (alpha = 0,
                                     lambda = c (2^c(-15:15),
                                     3^c(-15:15)))

models[[j]] <- train_model(formula = regression_formula,
                           train_data = whole_data_set,
                           method = "glmnet",
                           cv_type = "cv",
                           repeats = repeats,
                           tuneGrid = gridsearch_for_lambda,
                           number = k_of_cv)

col_index <- varImp(models[[j]], scale = FALSE)$importance %>%
                    mutate(names=row.names(.)) %>%
                    arrange(-Overall)
print (col_index)

  Overall            names
1  0.78253277        x_pos_3
2  0.63530210        x_pos_2
3  0.55951519        x_pos_4
4  0.45641051        x_pos_6
5  0.39452104        x_pos_5
6  0.13973329        y_pos_1
7  0.12288675        z_pos_4_pos_5
8  0.11361909        y_pos_2
...
col_index <- varImp(models[[j]], scale = FALSE, useModel=FALSE)$importance %>%
                    mutate(names=row.names(.)) %>%
                    arrange(-Overall)
  Overall            names
1  0.451045767   z_pos_4_pos_5
2  0.217605340   z_pos_3_pos_4
3  0.207219930   z_pos_5_pos_6
4  0.167938296   z_pos_3_pos_4
5  0.161774787   z_pos_1_pos_2
6  0.153217641   w_pos_2_pos_3
7  0.133689991   w_pos_4_pos_5
...
cor (whole_data_set[["response"]], z_pos_4_pos_5) 
-0.660

cor (whole_data_set[["response"]], x_pos_3) 
-0.142

I mentioned two ways I used varImp function (with useModel = TRUE, and useModel = FALSE). The outputs are completely different in both cases, and when I just do correlations of response to predictors (please see the correlation results at the bottom in the code), the second case (useModel = FALSE) seems more favorable case to trust.

I want to know how varImp works here (meaning that is it taking the weights assigned to the predictors after training the model?)

How can I list out the top 3 most important variables?

useModel=FALSE ... doesn't use the model and usually conducts some univariate importance score. When you print out the object it gives you information on the method.

For glmnet, it uses the absolute values of the model coefficients for the best regularization scheme determined by train. You can get the details using getModelInfo("glmnet")[[1]]$varImp

  • I looked into it and I think I am convinced. But just wondering why predict function was used to get the betas. Aren't betas implicit for a given lambda. May be this question I should ask to the glmnet developers. – satyanarayan rao Oct 18 '16 at 19:57

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.