I would like to perform a principal component regression (PCR), but feel a little confused about the rotation type to be used in the principal component analysis (PCA) step. First I perform a PCA to extract components, but I observe cross loadings after the extraction. Therefore, I perform a rotation (orthogonal or oblique) to overcome the cross-loadings issue and make components more interpretable. The thing is PCR is performed to overcome multicollinearity. As far as I have learned, with an orthogonal rotation technique, I assure that the regressors in PCR (at this time the selected components) will not be correlated, which is something good to avoid multicollinearity. However, oblique rotation techniques allow components to be correlated each other. In this case is it wrong to perform PCR with an oblique rotation technique? In other words, can PCR be performed only with unrotated or orthogonally rotated components?

  • $\begingroup$ Principal component analysis produces uncorrelated components. Why rotate at all if that is what you want? $\endgroup$ – Nick Cox Jul 9 '18 at 13:18
  • $\begingroup$ Thak you for your reply. I wanted to rotate, because the initial solution I got from the PCA had a group of cross-loadings. By applying rotation, my aim was to minimize (and totally eliminate, if I was lucky) the cross-loadings. $\endgroup$ – Onur Tekel Jul 14 '18 at 8:03
  • $\begingroup$ It's partly a matter of taste, but mine is that once you allow correlations between your predictors you're better off with a subset of your original predictors. $\endgroup$ – Nick Cox Jul 14 '18 at 8:05

A really nice overview, put in layman's terms, of rotation in any sort of Exploratory Factor Analysis can be found here: What is Rotating in Exploratory Factors Analysis?

The short answer to your question is that both orthogonal and oblique rotations both produce (virtually) identical results in the highly unlikely case that factors are perfectly uncorrelated. Historically, researchers using EFA have been drawn to orthogonal rotations because uncorrelated factors are easier to interpret. However, if factors are correlated (such as in human psychology, where identifiable factor groupings are likely going to have some correlations), then we likely want our model to reflect that.

In using PCR, instead of EFA, you do not need clearly defined or interpret-able factors, as you are using the method as a predictive technique. If the world of covariates indicates that there is some correlation between your factors, you likely want to model that. However, PCA is not the preferred method for modelling, though it is great for data compression and representation. Try PLS if you are interested in the modeling ability of your covariates.

  • 2
    $\begingroup$ There appears to be a tacit assumption here that when people say they are using PCA they are (should be) really thinking it's exploratory factor analysis (EFA). Where is that stated in the question? Much depends on tribe and taste, but often that is not the case at all. For example, Jolliffe's text on PCA is pretty emphatic that it doesn't have to be thought of as a species of factor analysis. $\endgroup$ – Nick Cox Jul 9 '18 at 13:22
  • $\begingroup$ Thanks again for your reply. The thing is, I really need interpretable factors. Because at the end of the PCR analysis I hope that the results would show me how the dependent variable is linked to the independent variables (which are produced via the PCA and made more interpretable using rotation in this case). $\endgroup$ – Onur Tekel Jul 14 '18 at 8:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.