3
$\begingroup$

I have a question about loading vectors and Eigen vectors. For the purposes of this discussion, a loading vector is an Eigen vector multiplied by the square root of the Eigen value.

  1. If the data is standardized (centered and divided by the standard deviation), is it correct to interpret the resulting loading vector coefficients as the correlation between the variables related to the loading vector coefficients and the loading vector in which the coefficients reside.
  2. If the data on which PCA analysis is performed is centered but we do not divide by the standard deviation, is it correct to interpret the loading vector coefficients as the covariance between the variables related to the loading vector coefficients and the loading vector in which the coefficients reside.
  3. If the data on which PCA analysis is performed is centered and the data is all in the same units (like TFIDF data for example), is it correct to interpret the resulting loading vector coefficients as the correlation between the variables related to the loading vector coefficients and the loading vector in which the coefficients reside.
  4. In any of the above cases, if we compute Eigen vectors instead of loading vectors, is it correct to interpret the Eigen vectors as containing the cosines of orthogonal transformation (rotation) of variables into principal components.
$\endgroup$
2
  • $\begingroup$ Will that bring more light? stats.stackexchange.com/q/143905/3277 $\endgroup$
    – ttnphns
    Commented Apr 28, 2020 at 1:10
  • $\begingroup$ I read that post before I posted the question and was not sure about my assertions above. $\endgroup$
    – Willard
    Commented Apr 28, 2020 at 2:04

1 Answer 1

2
$\begingroup$
  1. Yes. If the variables were standardized (that is alias to say that the PCA was based on correlation matrix), then loading values are correlations between the PCs and the variables. (All the p X m loadings are the correlations.)
  2. Likewise: Yes. If the variables were centered (that is alias to say that the PCA was based on covariance matrix), then loading values are covariances between the PCs and the variables.
  3. No. The loadings are covariances. Because there was no standardization, only was centering, the matrix implicitly analyzed is the covariance matrix, it has variances, not $1$s, on its diagonal.
  4. Yes. Eigenvector entries are the cosines of orthogonal transformation (rotation). It is so in both cases - when you analyze centered variables (covariance matrix) or standardized variables (correlation matrix). But the cosines will usually be different in the two cases because the rotation is different. That is to say, PCs are different when you base your PCA on standardized data and when you base it on centered data.
$\endgroup$
1
  • 1
    $\begingroup$ Thank you @ttnphns. This kind of high level information is extremely difficult to find without wandering very deep into heavy math. I appreciate you taking the time to answer this question. $\endgroup$
    – Willard
    Commented Apr 28, 2020 at 12:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.