# Which variables explain which PCA components, and vice versa?

Using this data:

head(USArrests)
nrow(USArrests)


I can do a PCA as thus:

plot(USArrests)
otherPCA <- princomp(USArrests)


I can get the new components in

otherPCA$scores  and the proportion of variance explained by components with summary(otherPCA)  But what if I want to know which variables are mostly explained by which principal components? And vice versa: is e.g. PC1 or PC2 mostly explained by murder? How can I do this? Can I say for instance that PC1 is 80% explained by murder or assault? I think the loadings help me here, but they show the directionality not the variance explained as i understand it, e.g. otherPCA$loadings

Comp.1 Comp.2 Comp.3 Comp.4
Murder                         0.995
Assault  -0.995
UrbanPop        -0.977 -0.201
Rape            -0.201  0.974

-
Note that the signs of the loadings are arbitrary. The three crime variables are all positively correlated with each other, but you would be smart to work that from the signs of the loadings above. –  Nick Cox Dec 3 '14 at 19:53
Unfortunately, I believe that the accepted answer to this question is incorrect. I posted my own answer below. –  amoeba Jan 15 at 22:59

You are right, the loadings can help you here. They can be used to compute the correlation between the variables and the principal components. Moreover, the sum of the squared loadings of one variable over all principal components is equal to 1. Hence, the squared loadings tell you the proportion of variance of one variable explained by one principal component.

The problem with princomp is, it only shows the "very high" loadings. But since the loadings are just the eigenvectors of the covariance matrix, one can get all loadings using the eigen command in R:

 loadings <- eigen(cov(USArrests))$vectors explvar <- loadings^2  Now, you have the desired information in the matrix explvar. - thank you random guy, could you possibly show me for example assault or urban pop we could do this? partly confused because there is only one correlation present in the matrix for assault – user1320502 Feb 18 '14 at 16:10 Sorry, I improved my answer and did not notice you commented my post already. assault loads with -0.995 on PC1. Thus, one can conclude after squaring this value PC1 explains 99% of the variance of the variable assault. After squaring the values of urban pop, you can conclude PC3 explains 4% and PC2 95.5% of the variance of urban pop. – random_guy Feb 18 '14 at 16:24 Doesn't OP ask about how much of the PCA can be attributed to a variable? Your answer is about how much of a variable can be explained by a CPA – Heisenberg Dec 3 '14 at 18:40 @amoeba Of course, they are not the correlations. Thanks for pointing that out. – random_guy Dec 3 '14 at 18:51 @Heisenberg Yes, however, I think the first part of the question goes in the direction I answered it and in the other part he got it the other way around. Nevertheless, what I wrote in response to his comment makes clear from what perspective I looked at it and I think that's okay for him because the answer was accepted. – random_guy Dec 3 '14 at 19:00 You can do a backwards or forwards stepwise variable selection predicting a component or a linear combination of components from their constituent variables. The$R^2$will be 1.0 at the first step if you use backwards stepdown. Even though stepwise regression is pretty much of a disaster when predicting$Y$it can work well when the prediction is mechanistic as is the case here. You can add or remove variables until you explain 0.8 or 0.9 (for example) of the information in the principal components. - Neat idea--thank you. – rolando2 Dec 4 '14 at 0:21 The US arrests data bundled with R are just an example here, but I note that the loadings calculations in the question come from a PCA of the covariance matrix. That's somewhere between arbitrary and nonsensical, as the variables are measured on different scales. Urban population looks like a percent. California is 91% and highest. The three crime variables appear to be number of arrests for crimes expressed relative to population size (presumably for some time period). Presumably it's documented somewhere whether it's arrests per 1000 or 10000 or whatever. The mean of the assault variable in the given units is about 171 and the mean murder is about 8. So, the explanation of your loadings is that in large part the pattern is an artefact: it depends on the very different variability of the variables. So, although there is sense in the data in that there are many more arrests for assaults than for murders, etc., that known (or unsurprising) fact dominates the analysis. This shows that, as any where else in statistics, you have to think about what you are doing in a PCA. If you take this further: 1. I'd argue that percent urban is better left out of the analysis. It's not a crime to be urban; it might of course serve proxy for variables influencing crime. 2. A PCA based on a correlation matrix would make more sense in my view. Another possibility is to work with logarithms of arrest rates, not arrest rates (all values are positive; see below). Note: @random_guy's answer deliberately uses the covariance matrix. Here are some summary statistics. I used Stata, but that's quite immaterial.  Variable | Obs Mean Std. Dev. Min Max -------------+-------------------------------------------------------- urban_pop | 50 65.54 14.47476 32 91 murder | 50 7.788 4.35551 .8 17.4 rape | 50 21.232 9.366384 7.3 46 assault | 50 170.76 83.33766 45 337  - I think that the accepted answer can be dangerously misleading (-1). There are at least four different questions mixed together in the OP. I will consider them one after another. • Q1. How much of the variance of a given PC is explained by a given original variable? How much of the variance of a given original variable is explained by a given PC? These two questions are equivalent and the answer is given by the square$r^2$of the correlation coefficient between the variable and the PC. If PCA is done on the correlations, then the correlation coefficient$r$is given (see here) by the corresponding element of the loadings. PC$i$is associated with an eigenvector$\mathbf V_i$of the correlation matrix and the corresponding eigenvalue$s_i$. A loadings vector$\mathbf L_i$is given by$\mathbf L_i = (s_i)^{1/2} \mathbf V_i$. Its elements are correlations of this PC with the respective original variables. Note that eigenvectors$\mathbf V_i$and loadings$\mathbf L_i$are two different things! In R, eigenvectors are confusingly called "loadings"; one should be careful: their elements are not the desired correlations. [The currently accepted answer in this thread confuses the two.] In addition, if PCA is done on covariances (and not on correlations), then loadings will also give you covariances, not correlations. To obtain correlations, one needs to compute them manually, following PCA. [The currently accepted answer is unclear about that.] • Q2. How much of the variance of a given original variable is explained by a given subset of PCs? How to select this subset to explain e.g.$80\%$of the variance? Because PCs are orthogonal (i.e. uncorrelated), one can simply add up individual$r^2$values (see Q1) to get the global$R^2$value. To select a subset, one can add PCs with the highest correlations ($r^2$) with a given original variable until the desired amount of explained variance ($R^2$) is reached. • Q3. How much of the variance of a given PC is explained by a given subset of original variables? How to select this subset to explain e.g.$80\%$of the variance? An answer to this question is not automatically given by PCA! E.g. if all original variables are very strongly inter-correlated with pairwise$r=0.9$, then correlations between the first PC and all the variables will be around$r=0.9$. One cannot add these$r^2$numbers to compute the proportion of variance of this PC explained by, say, five original variables (this would result in a nonsensical result$R^2 = 0.9\cdot0.9\cdot5>1$). Instead, one would need to regress this PC on these variables and obtain the multiple$R^2\$ value.

How to select a subset explaining given amount of variance, was suggested by @FrankHarrell (+1).

-