3
$\begingroup$

I've found this line of code to calculate predicted values from a ridge.lm model:

# Predict is not implemented so we need to do it ourselves
y.pred.ridge = scale(data.test[,1:8],center = F, scale = m.ridge$scales)%*% m.ridge$coef[,which.min(m.ridge$GCV)] + m.ridge$ym

Why center is set to FALSE? And why do I need to add the mean of $Y$ to the predicted values?

I thought $X$ values should be scaled before running a ridge regression, which implies the out of samples predictors should be centered and scaled? And scaling the predictors doesn't imply centering the outcomes of the regression, so why to add the mean of $Y$ to the predictions?

$\endgroup$

1 Answer 1

3
$\begingroup$

Why center is set to FALSE?

Because of information leakage. The idea with having a testing dataset is to mimic the situation you would have when using your model to make predictions on totally new data. Generally, you would not have the chance to center the data your model will see when it is put to use in a production environment. The correct procedure is to memorize the parameters needed to center the training data, then reuse them on any data you need to make predictions on. I believe the correct call to scale should be:

scale(data.test[,1:8], center = m.ridge$xm, scale = m.ridge$scales)

Which uses the fact that the m.ridge automatically centered and scaled its training data, and memorized the needed scaling parameters in m.ridge$xm and m.ridge$scales.

And why do I need to add the mean of Y to the predicted values?

Its the intercept term in the ridge regression. Because the intercept parameter is unpenalized, the intercept term is the mean of the response, just like in regular linear regression.

$\endgroup$
6
  • $\begingroup$ I was wondering whether lm.ridge automatically centers and scales. Did you get this information from the code? I checked the MASS book and couldn't find any documentation, not even a mention to lm.ridge(). $\endgroup$ May 14, 2015 at 16:56
  • $\begingroup$ Yah, I looked at the code. $\endgroup$ May 14, 2015 at 16:59
  • $\begingroup$ I see, by default the 'Inter' parameter is true, and that triggers centering. When predictors are centered, the intercept coefficient is the average predicted value when all predictors are 0, so Ym must be added. That (wrong) center=F line is very confusing. btw, what kind of scaling is {Xscale <- drop(rep(1/n, n) %*% X^2)^0.5}? $\endgroup$ May 14, 2015 at 22:02
  • $\begingroup$ (rep(1/n, n) %*% X^2)^.5 is a really clever matrix multiplication to compute the standard deviation of each column. The result is a 1 by n matrix, and the drop kills the dim attribute, making it a vector. This is then used in the following line X <- X/rep(Xscale, rep(n, p)) to broadcast divide this standard deviation and standardize the matrix. Kinda ridiculous : ) $\endgroup$ May 14, 2015 at 22:10
  • $\begingroup$ Why would centering conditional on the intercept? $\endgroup$ May 14, 2015 at 22:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.