Correlation between an original variable and a principal component

Question

I am looking at this link http://strata.uga.edu/8370/lecturenotes/principalComponents.html where it says

In interpreting the principal components, it is often useful to know the correlations of the original variables with the principal components. The correlation of variable $X_i$ and principal component $Y_j$ is

$$ r_{ij} = \sqrt{a_{ij}^2\times \mathrm{var}(Y_j)/s_{ii}}$$

where $a_{ij}$ is the $i$-th's variable principal component weight on on principal component $j$ and $Y_j$ is the $j$-th principal component score and I am not sure what $s_{ii}$ is.

I do a simple PCA on the iris dataset below and would like to calculate the correlation of sepal width and legth to PC score 1 and PC score 2 but what is $s_{ii}$?

The $\mathrm{var}(Y_j)$ is the variance of the principal component score. I have that in the VarofSCores object below. In that link the author says $S_Y$ is the varcov matrix of the scores so if $s_{ii}$ is the diagonal of $S_Y$ then that is the same value I have in VarOfSCores.

data(iris)
names(iris)
dat = data.frame(iris$Sepal.Width, iris$Sepal.Length)
pca= prcomp(dat)
PC = pca$rotation
VarOfScores = pca$sd^2
scores = pca$x

#correlation of sepal width to score 1
sqrt(PC[1,1]^2*  VarOfScores[1]/??   )

#correlation of sepal width to score 2
sqrt(PC[1,2]^2*  VarOfScores[2]/??   )

#correlation of sepal LENGTH to score 1
sqrt(PC[2,1]^2*  VarOfScores[1]/??   )

#correlation of sepal LENGTH to score 2
sqrt(PC[2,2]^2*  VarOfScores[2]/??   )

Also - Why does the author say "loadings" instead of "principal components". The eigenvectors are "principal components" not "loadings" and the data times the eigenvectors are "scores". I think that is poor terminology. See here http://www.cs.princeton.edu/courses/archive/spr08/cos424/scribe_notes/0424.pdf where the author states on on page 4 "V" is the principal components.

Answer 1

Correlation coefficient between variable $X_i$ and principal component $Y_j$ is given by $$r=v_{ij}\cdot \mathrm{std}(Y_j) / \mathrm{std}(X_i)=v_{ij}\cdot \sqrt{e_{j}} / \mathrm{std}(X_i),$$ where $v_{ij}$ is an $i$-th element of the $j$-th unit-length eigenvector of the covariance matrix, $e_j=\mathrm{var}(Y_j)$ is the corresponding eigenvalue which gives variance of this PC, and $\mathrm{std}(X_i)$ is standard deviation of $X_i$.

Please see my answer to How to find which variables are most correlated with the first principal component? for the derivation and additional explanations. Note that $v_{ij}\cdot \sqrt{e_{j}}$ are called "loadings" $L_{ij}$.

I think the formula in your quote says the same thing: $a_{ij}$ are the elements of the eigenvectors, $\mathrm{var}(Y_i)$ are the respective eigenvalues, and $s_{ii}$ stands for the diagonal elements of the original covariance matrix, i.e. for the variances of $\mathrm{var}(X_i)$.

Regarding terminology, please see What exactly is called "principal component" in PCA? and Loadings vs eigenvectors in PCA: when to use one or another?

Answer 2

Correlations with factors are called loadings. In PCA, eigenvectors can be scaled differently but if normalized to their eigenvalues, they are loadings (if memory serves). For other types of factor analysis, loadings and factors can differ but loadings are always refer to correlations.