While reading about PCA, I came across the following explanation:

Suppose we have a data set where each data point represents a single student's scores on a math test, a physics test, a reading comprehension test, and a vocabulary test.

We find the first two principal components, which capture 90% of the variability in the data, and interpret their loadings. We conclude that the first principal component represents overall academic ability, and the second represents a contrast between quantitative ability and verbal ability.

The text states that PC1 and PC2 loadings are $(0.5, 0.5, 0.5, 0.5)$ for PC1 and $(0.5, 0.5, -0.5, -0.5)$ for PC2, and offers the following explanation:

[T]he first component is proportional to average score, and the second component measures the difference between the first pair of scores and the second pair of scores.

I am not able to understand what this explanation means.

  • 12
    $\begingroup$ Somehow @ttnphns's answer goes into a lot of mathematical details, but I think the original question was really straightforward: why does the loadings vector for PC1 of (0.5, 0.5, 0.5, 0.5) mean that the first component is "proportional to average score"? Well, the answer is that the loadings are [proportional to the] coefficients in linear combination of original variables that makes up PC1. So your first PC1 is the sum of the all four variables times 0.5. Which means it's proportional to the average of the four variables. And similar with PC2. I think this answers the original question. $\endgroup$
    – amoeba
    Dec 6, 2014 at 14:36
  • 3
    $\begingroup$ @amoeba - Do you know how hard it is to come across such a simple explanation of loadings. Somehow, everywhere it is a mouthful of jargon bile all over me before i decide to move on to next explanation on google. Thank you! $\endgroup$ Apr 22, 2018 at 17:28

2 Answers 2


Loadings (which should not be confused with eigenvectors) have the following properties:

  1. Their sums of squares within each component are the eigenvalues (components' variances).
  2. Loadings are coefficients in linear combination predicting a variable by the (standardized) components.

You extracted 2 first PCs out of 4. Matrix of loadings $\bf A$ and the eigenvalues:

A (loadings)
         PC1           PC2
X1   .5000000000   .5000000000 
X2   .5000000000   .5000000000 
X3   .5000000000  -.5000000000 
X4   .5000000000  -.5000000000
    1.0000000000  1.0000000000

In this instance, both eigenvalues are equal. It is a rare case in real world, it says that PC1 and PC2 are of equal explanatory "strength".

Suppose you also computed component values, Nx2 matrix $\bf C$, and you z-standardized (mean=0, st. dev.=1) them within each column. Then (as point 2 above says), $\bf \hat {X}=CA'$. But, because you left only 2 PCs out of 4 (you lack 2 more columns in $\bf A$) the restored data values $\bf \hat {X}$ are not exact, - there is an error (if eigenvalues 3, 4 are not zero).

OK. What are the coefficients to predict components by variables? Clearly, if $\bf A$ were full 4x4, these would be $\bf B=(A^{-1})'$. With non-square loading matrix, we may compute them as $\bf B= A \cdot diag(eigenvalues)^{-1}=(A^+)'$, where diag(eigenvalues) is the square diagonal matrix with the eigenvalues on its diagonal, and + superscript denotes pseudoinverse. In your case:

1 0
0 1

B (coefficients to predict components by original variables):
    PC1           PC2
X1 .5000000000   .5000000000 
X2 .5000000000   .5000000000 
X3 .5000000000  -.5000000000 
X4 .5000000000  -.5000000000

So, if $\bf X$ is Nx4 matrix of original centered variables (or standardized variables, if you are doing PCA based on correlations rather than covariances), then $\bf C=XB$; $\bf C$ are standardized principal component scores. Which in your example is:

PC1 = 0.5*X1 + 0.5*X2 + 0.5*X3 + 0.5*X4 ~ (X1+X2+X3+X4)/4

"the first component is proportional to the average score"

PC2 = 0.5*X1 + 0.5*X2 - 0.5*X3 - 0.5*X4 = (0.5*X1 + 0.5*X2) - (0.5*X3 + 0.5*X4)

"the second component measures the difference between the first pair of scores and the second pair of scores"

In this example it appeared that $\bf B=A$, but in general case they are different.

Note: The above formula for the coefficients to compute component scores, $\bf B= A \cdot diag(eigenvalues)^{-1}$, is equivalent to $\bf B=R^{-1}A$, with $\bf R$ being the covariance (or correlation) matrix of variables. The latter formula comes directly from linear regression theory. The two formulas are equivalent within PCA context only. In factor analysis, they are not and to compute factor scores (which are always approximate in FA) one should rely on the second formula.

Related answers of mine:

More detailed about loadings vs eigenvectors.

How principal component scores and factor scores are computed.

  • 2
    $\begingroup$ If 2 components out of 4 account for 90% of variability how come their eigenvalues sum to 2? $\endgroup$
    – Nick Cox
    Dec 6, 2014 at 12:14
  • $\begingroup$ Nick, I believe this a question to the OP. He didn't give the data or covariance/correlation matrix. All we had from him is a (rather unrealistic) loading matrix of 2 first PCs. $\endgroup$
    – ttnphns
    Dec 6, 2014 at 12:23
  • 3
    $\begingroup$ Good point, @Nick, this is indeed not possible, as the total variance of a $4\times4$ correlation matrix must be $4$, so two PCs both with eigenvalues $1$ must account for $50\%$ of the variability. I am not explaining this to you, of course, but for other possible readers of this thread. The ttnphns's answer remains correct though (+1), we just have no other choice as to ignore the number $90\%$ reported by the OP. $\endgroup$
    – amoeba
    Dec 6, 2014 at 20:59

Although years are passed since last comments, I think the answer to the original question should be toward a more qualitative interpretation on how to read a "loadings matrix" (regardless of the assumptions we used to build it). If the vertical 'weights' are all the same (as in the original case for PC1 with all 0.5) it means that for PC1 all variables have same weight (0.5). If all variables have same weight it means that the 'scores' (in the PC matrix which is the matrix of the original data matrix projected along the eigenvectors) are proportional to th average of scores in the original data matrix. So PC1 tells me to evaluate a generic student based on its PC scores average (which is different in this case from the original average by a factor of 2x)

In relation to the second question, it's true that mathematically it's the difference between the scores of the two pairs, but the analysis of the PC2 tell us something about where the student is good or bad (as defined by PC1): so we can say that x1 and x2 move together and as much as x1 (and x2) is far from the average of its scores, x3 (and x4) is far from the average of its scores by the same amount in the opposite direction => as much more a student is good in math and phisics its scores in read/vocabulary decreas by the same amount.

So in conclusion by reading the Loadings Matrix we can formulate hypothesis that can be verified by looking in the 'PC matrix' and the 'scores': if you average the row for a generic student you can say how much he is good or bad by just looking at the value (don't forget that we decide that high/low means good/bad), if you then pick x1 (or x2) you can expect they are similar (both high or both low) and you can say from that if that student is good or bad in that subject, and by consequence you can expect he's bad or good in x3 (or x4).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.