4
$\begingroup$

I am looking at this link http://strata.uga.edu/8370/lecturenotes/principalComponents.html where it says

In interpreting the principal components, it is often useful to know the correlations of the original variables with the principal components. The correlation of variable $X_i$ and principal component $Y_j$ is

$$ r_{ij} = \sqrt{a_{ij}^2\times \mathrm{var}(Y_j)/s_{ii}}$$

where $a_{ij}$ is the $i$-th's variable principal component weight on on principal component $j$ and $Y_j$ is the $j$-th principal component score and I am not sure what $s_{ii}$ is.

I do a simple PCA on the iris dataset below and would like to calculate the correlation of sepal width and legth to PC score 1 and PC score 2 but what is $s_{ii}$?

The $\mathrm{var}(Y_j)$ is the variance of the principal component score. I have that in the VarofSCores object below. In that link the author says $S_Y$ is the varcov matrix of the scores so if $s_{ii}$ is the diagonal of $S_Y$ then that is the same value I have in VarOfSCores.

data(iris)
names(iris)
dat = data.frame(iris$Sepal.Width, iris$Sepal.Length)
pca= prcomp(dat)
PC = pca$rotation
VarOfScores = pca$sd^2
scores = pca$x

#correlation of sepal width to score 1
sqrt(PC[1,1]^2*  VarOfScores[1]/??   )

#correlation of sepal width to score 2
sqrt(PC[1,2]^2*  VarOfScores[2]/??   )

#correlation of sepal LENGTH to score 1
sqrt(PC[2,1]^2*  VarOfScores[1]/??   )

#correlation of sepal LENGTH to score 2
sqrt(PC[2,2]^2*  VarOfScores[2]/??   )

Also - Why does the author say "loadings" instead of "principal components". The eigenvectors are "principal components" not "loadings" and the data times the eigenvectors are "scores". I think that is poor terminology. See here http://www.cs.princeton.edu/courses/archive/spr08/cos424/scribe_notes/0424.pdf where the author states on on page 4 "V" is the principal components.

$\endgroup$
  • $\begingroup$ s is diagonal matrix of eigenvalues. Therefore, s_{ii} gives the eigenvalues of the sample covariance matrix. $\endgroup$ – ARAT Oct 24 '17 at 2:00
7
$\begingroup$

Correlation coefficient between variable $X_i$ and principal component $Y_j$ is given by $$r=v_{ij}\cdot \mathrm{std}(Y_j) / \mathrm{std}(X_i)=v_{ij}\cdot \sqrt{e_{j}} / \mathrm{std}(X_i),$$ where $v_{ij}$ is an $i$-th element of the $j$-th unit-length eigenvector of the covariance matrix, $e_j=\mathrm{var}(Y_j)$ is the corresponding eigenvalue which gives variance of this PC, and $\mathrm{std}(X_i)$ is standard deviation of $X_i$.

Please see my answer to How to find which variables are most correlated with the first principal component? for the derivation and additional explanations. Note that $v_{ij}\cdot \sqrt{e_{j}}$ are called "loadings" $L_{ij}$.

I think the formula in your quote says the same thing: $a_{ij}$ are the elements of the eigenvectors, $\mathrm{var}(Y_i)$ are the respective eigenvalues, and $s_{ii}$ stands for the diagonal elements of the original covariance matrix, i.e. for the variances of $\mathrm{var}(X_i)$.

Regarding terminology, please see What exactly is called "principal component" in PCA? and Loadings vs eigenvectors in PCA: when to use one or another?

$\endgroup$
2
$\begingroup$

Correlations with factors are called loadings. In PCA, eigenvectors can be scaled differently but if normalized to their eigenvalues, they are loadings (if memory serves). For other types of factor analysis, loadings and factors can differ but loadings are always refer to correlations.

$\endgroup$
  • $\begingroup$ Ok so what are your thoughts on what s_ii should be? $\endgroup$ – user3022875 Dec 29 '16 at 19:16
  • $\begingroup$ This applies only to PCA on correlation matrix. $\endgroup$ – amoeba Dec 29 '16 at 19:49
  • $\begingroup$ @amoeba I am doing pca on correlation matrix with my real data. This is just an example. Do you know what the author means for s_ii? $\endgroup$ – user3022875 Dec 29 '16 at 20:15
  • $\begingroup$ @user3022875 Yes. See my answer :) $\endgroup$ – amoeba Dec 29 '16 at 20:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.