I am interested in utilizing caret for making inferences on a particular data set. Is it possible to do the following:

  1. produce coefficients of a glmnet model I trained in caret. I would like to use glmnet because of the inherent feature selection as I do not believe glm has it?

  2. other than the ROC metric, is there another metric that I can utilize to asses fit of the model? Such as adjusted $R^2$?

The purpose of this analysis is to derive some inference on the effects of particular variables, rather than for prediction. I just like the caret package because it's been easy to work with thus far using matrices.

  • 3
    The caret package comes with a series of vignettes (and a JSS paper) that cover most of your questions. Could you indicate what precisely you mean by "derive some inference on the effect of particular variables?" – chl Aug 20 '13 at 6:43
  • Inference via the coefficients. I am reading through Applied Predictive Modeling to learn more about R and model building simultaneously. I had read the vignettes and the pdf, but there are just so many functions that it's hard to keep track of them all. Zach answered my question, however, so I am thankful. Thanks! – user2300643 Aug 20 '13 at 6:58
up vote 27 down vote accepted

Lets say your caret model is called "model". You can access the final glmnet model with model$finalModel. You can then call coef(model$finalModel), etc. You will have to select a value of lambda for which you want coefficients, such as coef(model$finalModel, model$bestTune$.lambda).

Take a look at the summaryFunction parameter for the trainControl function. It will allow you to specify any function you want to minimize (or maximize, see the maximize argument to train), given a predictor and a response.

It might be hard to get at adjusted R^2 in this way, but you could probably get R^2 or something similar.

  • 3
    Thank you, Zach. That was exactly it. Also, I want to thank you for your caretEnsemble package. Please keep up the good work. – user2300643 Aug 20 '13 at 6:23
  • @user2300643 No problem! I'm glad you're using the package. – Zach Aug 20 '13 at 13:41
  • 1
    In caret version 6.0.78, best tuned lambda is now: model$bestTune$lambda. – Harrison Mar 14 at 20:26

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.