14
$\begingroup$

Consider the following PCA biplot:

library(mvtnorm)
set.seed(1)
x  <- rmvnorm(2000, rep(0, 6), diag(c(5, rep(1,5))))
x  <- scale(x, center=T, scale=F)
pc <- princomp(x)
biplot(pc)

enter image description here

There are a bunch of red arrows plotted, what do they mean? I knew that the first arrow labelled with "Var1" should be pointing the most varying direction of the data-set (if we think them as 2000 data points, each being a vector of size 6). I also read from somewhere, the most varying direction should be the direction of the 1st eigen vector.

However, reading into the code of biplot in R. The line about the arrows is:

if(var.axes)
    arrows(0, 0, y[,1L] * 0.8, y[,2L] * 0.8, col = col[2L], 

Where y is the actually the loadings matrix, which is the eigenvector matrix. So it looks like the 1st arrow is actually pointing from (0, 0) to (y[1, 1], y[1, 2]). I understand that we are trying to plot a high dimensional arrow onto a 2D plane. That's why we are taking the 1st and 2nd element of the y[1, ] vector. However what I don't understand is:

Shouldn't the 1st eigenvector direction be the vector denoted by y[, 1], instead of y[1, ]? (Again, here y is the eigenvector matrix, obtained by PCA or by eigendecomposition of t(x) %*% x.) i.e. the eigenvectors should be column vectors, not those horizontal vectors.

Even though we are plotting them on 2D plane, we should draw the 1st direction to be from (0, 0) pointing to (y[1, 1], y[2, 1])?

$\endgroup$
  • 2
    $\begingroup$ Also consider trying to utilize the formatting available to questions on the site, see the advanced markdown help. I see none of your previous questions have answers, and if you edit them to be slightly more readable they will bump to the top of the list and may attract answers. $\endgroup$ – Andy W Apr 25 '12 at 12:01
  • $\begingroup$ @Andy W Excellent references $\endgroup$ – conjugateprior Apr 25 '12 at 13:27
  • $\begingroup$ @Andy W: After reading your excellent references, I still think the R code biplot.princomp has a bug: the loading matrix (eigenvector matrix) should be transposed before being sent into biplot.princomp... Any thoughts? $\endgroup$ – Luna Apr 25 '12 at 16:28
  • $\begingroup$ It looks like there is no bug. $\endgroup$ – chl Apr 28 '12 at 20:17
  • $\begingroup$ @Andy W: do you want to put your comments into an "answer" so I can accept your answer? That's a great one! Thanks so much! $\endgroup$ – Luna Apr 29 '12 at 16:24
9
$\begingroup$

Well it appears Kevin Wright should be given most of the credit to try to help explain the confusion (from R-help mail list);

The arrows are not pointing in the most-varying direction of the data. The principal components are pointing in the most-varying direction of the data. But you are not plotting the data on the original scale, you are plotting the data on the rotated scale, and thus the horizontal axis is the most-varying direction of the data.

The arrows are pointing in the direction of the variables, as projected into the 2-d plane of the biplot.

There is no bug.

Kevin Wright

Michael Greenacre has a very excellent free online book about biplots, Biplots in Practice, and simply reading the first chapter should help motivate where the coordinates of the arrows are taken from. There are also several other questions on the site that are similar and you may be interested in, see Interpretation of biplots in principal components analysis in R and Interpretation of MDS factor plot for two examples. Also look through the questions with biplot in search on the site, as there are a few more of potential interest (it appears maybe even making a biplot tag would be useful at this point given the number of questions it has come up in).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.