What do the arrows in a PCA biplot mean?

Question

Consider the following PCA biplot:

library(mvtnorm)
set.seed(1)
x  <- rmvnorm(2000, rep(0, 6), diag(c(5, rep(1,5))))
x  <- scale(x, center=T, scale=F)
pc <- princomp(x)
biplot(pc)

There are a bunch of red arrows plotted, what do they mean? I knew that the first arrow labelled with "Var1" should be pointing the most varying direction of the data-set (if we think them as 2000 data points, each being a vector of size 6). I also read from somewhere, the most varying direction should be the direction of the 1st eigen vector.

However, reading into the code of biplot in R. The line about the arrows is:

if(var.axes)
    arrows(0, 0, y[,1L] * 0.8, y[,2L] * 0.8, col = col[2L], 

Where y is the actually the loadings matrix, which is the eigenvector matrix. So it looks like the 1st arrow is actually pointing from (0, 0) to (y[1, 1], y[1, 2]). I understand that we are trying to plot a high dimensional arrow onto a 2D plane. That's why we are taking the 1st and 2nd element of the y[1, ] vector. However what I don't understand is:

Shouldn't the 1st eigenvector direction be the vector denoted by y[, 1], instead of y[1, ]? (Again, here y is the eigenvector matrix, obtained by PCA or by eigendecomposition of t(x) %*% x.) i.e. the eigenvectors should be column vectors, not those horizontal vectors.

Even though we are plotting them on 2D plane, we should draw the 1st direction to be from (0, 0) pointing to (y[1, 1], y[2, 1])?

Answer 1

Well it appears Kevin Wright should be given most of the credit to try to help explain the confusion (from R-help mail list);

The arrows are not pointing in the most-varying direction of the data. The principal components are pointing in the most-varying direction of the data. But you are not plotting the data on the original scale, you are plotting the data on the rotated scale, and thus the horizontal axis is the most-varying direction of the data.

The arrows are pointing in the direction of the variables, as projected into the 2-d plane of the biplot.

There is no bug.

Kevin Wright

Michael Greenacre has a very excellent free online book about biplots, Biplots in Practice, and simply reading the first chapter should help motivate where the coordinates of the arrows are taken from. There are also several other questions on the site that are similar and you may be interested in, see Interpretation of biplots in principal components analysis in R and Interpretation of MDS factor plot for two examples. Also look through the questions with biplot in search on the site, as there are a few more of potential interest (it appears maybe even making a biplot tag would be useful at this point given the number of questions it has come up in).