Interpretation of biplots in principal components analysis

Question

I came across this nice tutorial: A Handbook of Statistical Analyses Using R. Chapter 13. Principal Component Analysis: The Olympic Heptathlon on how to do PCA in R language. I don't understand the interpretation of Figure 13.3:

So I am plotting first eigenvector vs the second eigenvector. What does that mean? Suppose eigenvalue corresponding to first eigenvector explains 60% of variation in data set and second eigenvalue-eigenvector explain 20% of variation. What does it mean to plot these against each other?

Answer 1

PCA is one of the many ways to analyse the structure of a given correlation matrix. By construction, the first principal axis is the one which maximizes the variance (reflected by its eigenvalue) when data are projected onto a line (which stands for a direction in the $p$-dimensional space, assuming you have $p$ variables) and the second one is orthogonal to it, and still maximizes the remaining variance. This is the reason why using the first two axes should yield the better approximation of the original variables space (say, a matrix $X$ of dim $n \times p$) when it is projected onto a plane.

Principal components are just linear combinations of the original variables. Therefore, plotting individual factor scores (defined as $Xu$, where $u$ is the vector of loadings of any principal component) may help to highlight groups of homogeneous individuals, for example, or to interpret one's overall scoring when considering all variables at the same time. In other words, this is a way to summarize one's location with respect to his value on the $p$ variables, or a combination thereof. In your case, Fig. 13.3 in HSAUR shows that Joyner-Kersee (Jy-K) has a high (negative) score on the 1st axis, suggesting he performed overall quite good on all events. The same line of reasoning applies for interpreting the second axis. I take a very short look at the figure so I will not go into details and my interpretation is certainly superficial. I assume that you will find further information in the HSAUR textbook. Here it is worth noting that both variables and individuals are shown on the same diagram (this is called a biplot), which helps to interpret the factorial axes while looking at individuals' location. Usually, we plot the variables into a so-called correlation circle (where the angle formed by any two variables, represented here as vectors, reflects their actual pairwise correlation, since the cosine of the angle between pairs of vectors amounts to the correlation between the variables.

I think, however, you'd better start reading some introductory book on multivariate analysis to get deep insight into PCA-based methods. For example, B.S. Everitt wrote an excellent textbook on this topic, An R and S-Plus^® Companion to Multivariate Analysis, and you can check the companion website for illustration. There are other great R packages for applied multivariate data analysis, like ade4 and FactoMineR.

Answer 2

The plot is showing:

• the score of each case (i.e., athlete) on the first two principal components
• the loading of each variable (i.e., each sporting event) on the first two principal components.

The left and bottom axes are showing [normalized] principal component scores; the top and right axes are showing the loadings.

In general it assumes that two components explain a sufficient amount of the variance to provide a meaningful visual representation of the structure of cases and variables.

You can look to see which events are close together in the space. Where this applies, this may suggest that athletes who are good at one event are likely also to be good at the other proximal events. Alternatively you can use the plot to see which events are distant. For example, javelin appears to be bit of an outlier and a major event defining the second principal component. Perhaps a different kind of athlete is good at javelin than is good at most of the other events.

Of course, more could be said about substantive interpretation.