I did a lot of reading in this blog and elsewhere about PCA, SVD, loadings etc. But I still don't understand why loadings, which represent correlations between principal components and the original variables, are mathematically defined by

loadings = eigenvector * square root (eigenvalue)

It seems I just can't grasp it. Could somebody please explain me the mathematics behind it?

  • 3
    $\begingroup$ This is only true if all the original variables were standardized prior to PCA. You can find a mathematical explanation e.g. in the beginning of my answer here stats.stackexchange.com/questions/104306 $\endgroup$ – amoeba Dec 20 '18 at 13:13
  • 1
    $\begingroup$ Since a correlation is a number (between -1 and 1) and your definition of "loading" is a vector whose components could have arbitrarily large values, it isn't plausible to describe your loading as "representing correlation." $\endgroup$ – whuber Dec 20 '18 at 14:28
  • $\begingroup$ @whuber The word "correlation" should be in plural. I edited. Other than that, the question makes total sense. $\endgroup$ – amoeba Dec 20 '18 at 14:59
  • $\begingroup$ @amoeba Thank a lot for your answer. The link you posted helped my to understand loadings. But I am still struggeling with the mathematics. In your linked answer there is this equation to compute cross-covariance matrix between original variables and standardized PCs. And I think this is exactly the answer to my question. Its starts with 1/N-1 * X(transposed)* (squareroot(N-1)*U) Where does this formular comes from? I also don't understand the first transformation of the equation . It would really great if you could explain this equation to me, especially the first step. $\endgroup$ – Concetta Dec 20 '18 at 15:09
  • $\begingroup$ Cross-covariance matrix between matrices A and B (assuming both have centered columns) is A.transposed * B / n. Can you be more specific as to what you don't understand? Do you know what covariance is? Can you follow these matrix operations? I don't know what level of explanation you need. $\endgroup$ – amoeba Dec 20 '18 at 15:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.