I have a question about loading vectors and Eigen vectors. For the purposes of this discussion, a loading vector is an Eigen vector multiplied by the square root of the Eigen value.

1. If the data is standardized (centered and divided by the standard deviation), is it correct to interpret the resulting loading vector coefficients as the correlation between the variables related to the loading vector coefficients and the loading vector in which the coefficients reside.
2. If the data on which PCA analysis is performed is centered but we do not divide by the standard deviation, is it correct to interpret the loading vector coefficients as the covariance between the variables related to the loading vector coefficients and the loading vector in which the coefficients reside.
3. If the data on which PCA analysis is performed is centered and the data is all in the same units (like TFIDF data for example), is it correct to interpret the resulting loading vector coefficients as the correlation between the variables related to the loading vector coefficients and the loading vector in which the coefficients reside.
4. In any of the above cases, if we compute Eigen vectors instead of loading vectors, is it correct to interpret the Eigen vectors as containing the cosines of orthogonal transformation (rotation) of variables into principal components.
• Will that bring more light? stats.stackexchange.com/q/143905/3277 Commented Apr 28, 2020 at 1:10
• I read that post before I posted the question and was not sure about my assertions above. Commented Apr 28, 2020 at 2:04

1. Yes. If the variables were standardized (that is alias to say that the PCA was based on correlation matrix), then loading values are correlations between the PCs and the variables. (All the p X m loadings are the correlations.)
3. No. The loadings are covariances. Because there was no standardization, only was centering, the matrix implicitly analyzed is the covariance matrix, it has variances, not $$1$$s, on its diagonal.