Let's say I have X variables that I want to use to predict Y (Response). Should I provide all X variables to perform Principal components regression (PCR) or should I provide only the significantly correlated variables (obtained from finding partial correlation for controlling confounding factors using Spearman correlation) for predicting the response using PCR? Would the results change? I'm a biologist and a novice in this field. Please help.


First, it's not clear that you should use principal components regression; partial least squares is often a better method when you have a lot of independent variables that you want to combine into a few in order to use them in regression.

Second, no, you shouldn't filter out the variables based on correlation with the DV, you should use any IVs that you are interested in. For one thing, it could be that some combination of variables is a good predictor.

  • 1
    $\begingroup$ Was going to write pretty much the same answer. I'd also mention ridge regression as a superior alternative. $\endgroup$ – Matthew Drury May 11 '17 at 14:15
  • $\begingroup$ @MatthewDrury Hmm. I hadn't heard of using ridge regression here. I thought that was mostly used when there was collinearity. Why does it work well here? $\endgroup$ – Peter Flom May 11 '17 at 14:54
  • $\begingroup$ @Peter Flom Hmm. I forgot to mention that my variables are not independent. They are also correlated to each other. So if I have to make them uncorrelated, I need to perform a PCA. Correct me if I am wrong. Also, if I don't pre-process the data using highly correlated variables to the response, won't I end up in choosing combination of variables that might not have any effect in reality and behave like an artifact. I might end up getting variance explained by inappropriate uncorrelated variables. $\endgroup$ – Abhishek Subramanian May 12 '17 at 19:17
  • $\begingroup$ Ridge regression works well with correlated variables - I think that's what @MatthewDrury meant and I misunderstood. With ridge regression you don't need PCA. And partial least squares also works with correlated variables. Or you could drop some variables. $\endgroup$ – Peter Flom May 13 '17 at 14:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.