1
$\begingroup$

I have tuned a glm net model with caret using the train function. I am trying to extract the coefficients and standard errors of those coefficients for the best tuned model. Following this CV post I figured out how extract the coefficients. As such, I use

coef <- as.matrix(coef(model$finalModel, model$bestTune$lambda))

to extract the coefficients. However, I cannot figure out how to get the standard errors. I tried se.coef (se.coef <- as.matrix(se.coef(model$finalModel, model$bestTune$lambda)) from the "arm" library but it threw the following error:

Error in as.matrix(se.coef(model$finalModel, model$bestTune$lambda)) : 
  error in evaluating the argument 'x' in selecting a method for function 'as.matrix': Error in (function (classes, fdef, mtable)  : 
  unable to find an inherited method for function ‘se.coef’ for signature ‘"lognet"’

Could anyone add any insight please? I understand that the se.coef may not work with the train function. However, is there another way to obtain the standard errors for the coefficients?

Thanks.

$\endgroup$
1
$\begingroup$

Perhaps you could bootstrap your data? For example:

library(caret)
library(boot)

n <- 500L

x1 <- rnorm(n, 2.0, 0.5)
x2 <- rnorm(n, -1.0, 2)
y <- factor(rbinom(n, 1L, plogis(-0.6 + 1.0 * x1 - 0.8 * x2)))

dat <- data.frame(y, x1, x2)

caretMod <- train(y ~ ., 
        data = dat, 
        method = "glmnet",
        trControl = trainControl(method = "CV"))

bootSamples <- boot(dat, function(data, idx) {
            bootstrapData <- data[idx, ]

            bootstrapMod <- train(y ~ ., 
                    data = bootstrapData, 
                    method = "glmnet", 
                    trControl = trainControl(method = "none"),
                    tuneGrid = caretMod$bestTune)

            as.vector(coef(bootstrapMod$finalModel, caretMod$bestTune$lambda))
        }, 100L)

Bootstrap Statistics :
      original        bias    std. error
t1*  0.1771481 -0.0232223436  0.27816559
t2*  0.5062797  0.0101922882  0.11641296
t3* -0.3857333  0.0002638111  0.02492466
$\endgroup$
  • $\begingroup$ This is definitely very helpful. It certainly provides a good estimate. Does this mean that there is no std errors stored in the train object like there are coefficients? Thanks again! $\endgroup$ – TSW Apr 23 '15 at 15:52
  • 1
    $\begingroup$ Yes, I don't think glmnet gives any indications into the standard errors of the coefficients. $\endgroup$ – Jeff Apr 23 '15 at 17:34
  • 1
    $\begingroup$ Having done further research, it does appear that bootstrapping is only real way to estimate the std. errors for a penalized regression. Further, it is unclear how meaningful the std. errors are in this case. Thank you the help and the code! $\endgroup$ – TSW Apr 24 '15 at 16:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.