I keep reading that the angle between loadings in a PCA plot indicates some degree of correlation between the loadings (presumably lower angles lead to higher correlation, and vice versa).

I don't fully understand this. Say we have two features, $X$ and $Y$, where $Y=cX + \epsilon$, that is, Y is correlated with X plus some small amount of random noise. If I run a PCA on this, I will get one large principal component and one small. If the loading vectors correspond to the features themselves, then I will simply get the projection of the original axes onto the principal components; these projected vectors don't seem to have a low angle between them.

So what exactly is the relationship between the angles between loading vectors and their degree of correlation? To me, a low angle between two loadings seems to only indicate that two particular features have the same level of contribution towards a particular direction of variance, but I'm not sure if there's more than just this.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.