7
$\begingroup$

I am looking at this link http://strata.uga.edu/8370/lecturenotes/principalComponents.html where it says

In interpreting the principal components, it is often useful to know the correlations of the original variables with the principal components. The correlation of variable $X_i$ and principal component $Y_j$ is

$$ r_{ij} = \sqrt{a_{ij}^2\times \mathrm{var}(Y_j)/s_{ii}}$$

where $a_{ij}$ is the $i$-th's variable principal component weight on on principal component $j$ and $Y_j$ is the $j$-th principal component score and I am not sure what $s_{ii}$ is.

I do a simple PCA on the iris dataset below and would like to calculate the correlation of sepal width and legth to PC score 1 and PC score 2 but what is $s_{ii}$?

The $\mathrm{var}(Y_j)$ is the variance of the principal component score. I have that in the VarofSCores object below. In that link the author says $S_Y$ is the varcov matrix of the scores so if $s_{ii}$ is the diagonal of $S_Y$ then that is the same value I have in VarOfSCores.

data(iris)
names(iris)
dat = data.frame(iris$Sepal.Width, iris$Sepal.Length)
pca= prcomp(dat)
PC = pca$rotation
VarOfScores = pca$sd^2
scores = pca$x

#correlation of sepal width to score 1
sqrt(PC[1,1]^2*  VarOfScores[1]/??   )

#correlation of sepal width to score 2
sqrt(PC[1,2]^2*  VarOfScores[2]/??   )

#correlation of sepal LENGTH to score 1
sqrt(PC[2,1]^2*  VarOfScores[1]/??   )

#correlation of sepal LENGTH to score 2
sqrt(PC[2,2]^2*  VarOfScores[2]/??   )

Also - Why does the author say "loadings" instead of "principal components". The eigenvectors are "principal components" not "loadings" and the data times the eigenvectors are "scores". I think that is poor terminology. See here http://www.cs.princeton.edu/courses/archive/spr08/cos424/scribe_notes/0424.pdf where the author states on on page 4 "V" is the principal components.

$\endgroup$
1
  • $\begingroup$ s is diagonal matrix of eigenvalues. Therefore, s_{ii} gives the eigenvalues of the sample covariance matrix. $\endgroup$
    – ARAT
    Oct 24, 2017 at 2:00

2 Answers 2

9
$\begingroup$

Correlation coefficient between variable $X_i$ and principal component $Y_j$ is given by $$r=v_{ij}\cdot \mathrm{std}(Y_j) / \mathrm{std}(X_i)=v_{ij}\cdot \sqrt{e_{j}} / \mathrm{std}(X_i),$$ where $v_{ij}$ is an $i$-th element of the $j$-th unit-length eigenvector of the covariance matrix, $e_j=\mathrm{var}(Y_j)$ is the corresponding eigenvalue which gives variance of this PC, and $\mathrm{std}(X_i)$ is standard deviation of $X_i$.

Please see my answer to How to find which variables are most correlated with the first principal component? for the derivation and additional explanations. Note that $v_{ij}\cdot \sqrt{e_{j}}$ are called "loadings" $L_{ij}$.

I think the formula in your quote says the same thing: $a_{ij}$ are the elements of the eigenvectors, $\mathrm{var}(Y_i)$ are the respective eigenvalues, and $s_{ii}$ stands for the diagonal elements of the original covariance matrix, i.e. for the variances of $\mathrm{var}(X_i)$.

Regarding terminology, please see What exactly is called "principal component" in PCA? and Loadings vs eigenvectors in PCA: when to use one or another?

$\endgroup$
2
$\begingroup$

Correlations with factors are called loadings. In PCA, eigenvectors can be scaled differently but if normalized to their eigenvalues, they are loadings (if memory serves). For other types of factor analysis, loadings and factors can differ but loadings are always refer to correlations.

$\endgroup$
4
  • $\begingroup$ Ok so what are your thoughts on what s_ii should be? $\endgroup$ Dec 29, 2016 at 19:16
  • $\begingroup$ This applies only to PCA on correlation matrix. $\endgroup$
    – amoeba
    Dec 29, 2016 at 19:49
  • $\begingroup$ @amoeba I am doing pca on correlation matrix with my real data. This is just an example. Do you know what the author means for s_ii? $\endgroup$ Dec 29, 2016 at 20:15
  • $\begingroup$ @user3022875 Yes. See my answer :) $\endgroup$
    – amoeba
    Dec 29, 2016 at 20:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.