Orthogonalization via PCA and ridge regression are two common methods to account for multicollinearity for linear regression models. When would you use one over the other?
4
-
1$\begingroup$ This sounds suspiciously like a homework problem. $\endgroup$– ZachCommented Jul 19, 2012 at 17:10
-
$\begingroup$ This is not a homework problem, and I was hoping for something other than whatever works best empirically. Correct me if I'm wrong, but I believe ridge regression encodes a multivariate mean-0 normal prior on the regression parameters ... assuming that you have reason to believe the regression parameters should follow this prior, is there any a priori reason to prefer one to the other? $\endgroup$– nanCommented Jul 19, 2012 at 18:11
-
$\begingroup$ @nan, you are correct - ridge regression is equivalent to a fitting a Guassian linear model with a Gaussian prior on the $\beta$s. Your second sentence confuses me though - if you have reason to believe that this is a good prior, then, of course, you do have an a priori reason to prefer ridge regression. Maybe I've misunderstood your query. $\endgroup$– MacroCommented Jul 19, 2012 at 18:38
-
$\begingroup$ you may also find this thread useful. $\endgroup$– MacroCommented Jul 19, 2012 at 18:47
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
When the cross-validated error of one method is lower than the other. I would also look into lasso regression and elastic net regression.
