0
$\begingroup$

I am not quite sure how to interpret the output of this code:

 coef(ridge_model, s = cv.glmnet(model, y, k=k)$lambda.min)

ridge_model is the output of glmnet()

What role does the argument 's' play?

Output:

7 x 1 sparse Matrix of class "dgCMatrix"
                    1
(Intercept) 86.825637
(Intercept)  .       
x1           3.924821
x2           9.816783
x3          11.770995
x4B         22.385858
x4C         -6.438195
  1. My confusion is in understanding how the coef() function works. ridge_model is
    the output of glmnet() so it represents the fitted model for different lambda values. Each lambda would have its set of coefficients.
  2. Then there is the cv.glmnet() that gives the k-fold cross validation output and gives the minimum lambda value. We are giving this lambda as an input to the 's' argument.
  3. How would this then affect the ridge model which already has its lambda values? coef(ridge_model, s = cv.glmnet(model, y, k=k)$lambda.min)
$\endgroup$
1
  • 1
    $\begingroup$ This is really more of a statistical question rather than a programming question; s is the strength of the penalty/shrinkage term. See the description of the s parameter in ?coef.glmnet ... $\endgroup$
    – Ben Bolker
    Feb 13, 2016 at 15:38

2 Answers 2

2
$\begingroup$

This smells incorrect, you probably wanted:

fit <- cv.glmnet(model, y, k=k)
coef(fit, "lambda.min")

which will return the coefficients using the internal fit from the cross validation.

Unless ridge_model has the same predictors, weights, mixing parameter, etc, plugging in a penalty parameter from one model into another seems odd; but if that were the same, ridge_model would be the same as fit$glmnet.fit above and redundant.

$\endgroup$
3
  • $\begingroup$ Neal, what i wanted to do was create two separate functions. One for ridge regression and the other for cross validation. The model obtained from ridge regression had to be used for cross validation. $\endgroup$
    – RDPD
    Feb 13, 2016 at 16:04
  • $\begingroup$ I wasn't quite sure how to build such functions. $\endgroup$
    – RDPD
    Feb 13, 2016 at 16:05
  • 3
    $\begingroup$ If you are interested in writing your own k-fold cross validation, you should look at how cross validation is implemented in cv.glmnet or the caret package's source code. If you just want to use cv.glmnet to fit a model, it already does the right thing for most cases and you shouldn't need to reimplement more cross-validation code on top of it. $\endgroup$
    – Neal Fultz
    Feb 13, 2016 at 16:46
0
$\begingroup$

A summary of the glmnet path at each step is displayed if we just enter the object name or use the print function:

  print(fit)

  ## 
  ## Call:  glmnet(x = x, y = y) 
  ## 
  ##       Df   %Dev  Lambda
  ##  [1,]  0 0.0000 1.63000
  ##  [2,]  2 0.0553 1.49000
  ##  [3,]  2 0.1460 1.35000
  ##  [4,]  2 0.2210 1.23000

It shows from left to right the number of nonzero coefficients (Df), the percent (of null) deviance explained (%dev) and the value of λ

(Lambda). Although by default glmnet calls for 100 values of lambda the program stops early if `%dev% does not change sufficently from one lambda to the next (typically near the end of the path.)

We can obtain the actual coefficients at one or more λ

’s within the range of the sequence:

  coef(fit,s=0.1)

  ## 21 x 1 sparse Matrix of class "dgCMatrix"
  ##                     1
  ## (Intercept)  0.150928
  ## V1           1.320597
  ## V2           .       
  ## V3           0.675110
  ## V4           .       
  ## V5          -0.817412

Here is the original explanation for more information by Hastie

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.