In How to Deal with Multicollinearity?, the top comment in Aaron's answer says "These independent variables are now uncorrelated. Very low eigenvalues also indicate high degrees of multicollinearity in the original data.."

Why are the variables now uncorrelated, and why do very low eigenvalue indicate high degrees of multicollinearity?

  • 1
    $\begingroup$ Hmm I'm not sure if that comment on the independent variables being uncorrelated is correct. The principal components corresponding to distinct singular values are orthogonal, which implies linear independency, but if the principal components are not centered, then it doesn't imply that they're uncorrelated. Centered orthogonality implies the variables are correlated, since covariance between $X,Y$ is defined as $E[(X - \mu_X)(Y - \mu_Y)]$. $\endgroup$ – anonuser01 Aug 8 '20 at 18:59
  • $\begingroup$ My post at stats.stackexchange.com/a/74328/919 answers your second question. The first ("why are the variables now uncorrelated?") is basic to PCA; you can find explanations in many PCA-related threads on this site. $\endgroup$ – whuber Aug 8 '20 at 19:04
  • $\begingroup$ @whuber I will check it out. As lamanon said above the principal components are orthogonal, but may not be uncorrelated if they're not centered. Are the principal components centered? I know that typically before we perform PCA, we perform centering on the original variables, but does this result in the principal components being centered too? If not, then it still may be correlated? $\endgroup$ – user5965026 Aug 8 '20 at 19:07
  • $\begingroup$ Yes. The original means are mapped to the principal component space, so the zero mean is preserved. But if you don't center the data before hand, then there's no guarantee of uncorrelation. There is only guarantee of orthogonality of the eigenvectors corresponding to the distinct eigenvalues of the covariance matrix. $\endgroup$ – anonuser01 Aug 8 '20 at 19:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.