5
$\begingroup$

I'm currently running a ridge regression in R using the glmnet package, however, I recently ran into a new problem and was hoping for some help in interpreting my results. My data can be found here: https://www.dropbox.com/sh/hpxu3t0vqkrzfgf/AAB6F-yMYMfuI5E__gfDuW6sa?dl=0

My data consists of a 26531x428 observation matrix x and a 26531x1 response vector y. I am attempting to determine the optimal value of lambda.min, and when I run the code

> lambda=cv.glmnet(x=x,y=y,weights=weights,alpha=0,nfolds=10,standardize=FALSE)

I get

$lambda.min [1] 2.123479 $lambda.1se [1] 619.0054

which are results I would expect. However, I would like to add a slight tweak to this regression. I have prior knowledge of each of my 428 coefficients, and instead of shrinking each coefficient towards 0, as is the default with ridge regression, I would like to shrink each coefficient towards a specific value other than 0. After reaching out to Dr. Trevor Hastie, one of the creators of glmnet, he told me that this could be achieved by running the same code after substituting y with y2, where y2 = y - x%*%d and d is a 428x1 vector of coefficient priors. He said to then add d to my new coefficients, which would give me my prior-informed coefficients. After rerunning the code

> lambda=cv.glmnet(x=x,y=y2,weights=weights,alpha=0,nfolds=10,standardize=FALSE)

I unfortunately get

$lambda.min [1] 220.3026 $lambda.1se [1] 220.3026

The results of plot(lambda) look like this lambda plot

Does anyone know why glmnet can't find a suitable lambda.min? Could it be because my vector of priors contains estimates that are too far off? Any help would be greatly appreciated!

$\endgroup$

1 Answer 1

3
$\begingroup$

Glmnet is returning a very large optimal regularization parameter, i.e., it is regularizing away all of your coefficients. It looks like glmnet is telling you that, after accounting for your prior (or offset) coefficients, what is left is noise. That is, you already offset the correct coefficients, and the model is just validating that.

$\endgroup$
4
  • $\begingroup$ Thanks for the quick response. So, if I understand you correctly, does that mean I should be ok using the coefficient estimates from the prior as my coefficients and ignore the glmnet results altogether? $\endgroup$
    – dwm8
    Sep 28, 2015 at 16:57
  • $\begingroup$ It looks to me from that plot that the coefficients from glmnet (at the optimal lambda) should be all zero, is that correct? $\endgroup$ Sep 28, 2015 at 17:00
  • $\begingroup$ Yes, the coefficients at lambda.min are all zero, so when I add that coefficient vector to the prior coefficient vector, it obviously is just the prior coefficient vector. Does that change your interpretation of the results at all? $\endgroup$
    – dwm8
    Sep 28, 2015 at 17:03
  • $\begingroup$ No, I just asked that to validate and double check. So yah, I think our interpretation is solid then: you've validated that your prior coefficient vector is, as far as glmnet can tell, completely correct. $\endgroup$ Sep 28, 2015 at 17:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.