I'm learning PCA in R language. I met two problems right now that I don't understand.

  1. I am performing a PCA analysis in R on a 318×17 dataset using some custom code. I take eigen function in R to find eigenvalues and eigenvectors. But my 1st and 3rd eigenvectors are of the opposite sign to my handbook. My second eigenvectors is almost the same.

  2. I know that given a square matrix A, the condition that characterizes an eigenvalue, $\lambda$, is the existence of a nonzero vector $x$ such that $Ax=\lambda x$; this equation can be rewritten as follows: $(A - \lambda)x=0$.

    Now I calculate covariance of my data and have eigenvalues. I want to solve this linear combination equation to find $x$ and compare with initial eigenvectors. When I take solve function in R, my $x$ vector is always zero.

Here are my questions: Why the sign is different? How to use solve function in R to find a non-zero vector $x$?

  • Regarding the first part of your question, I encountered the same wrinkle when working on this post. It came to no surprise that I could have followed through the rest of the derivation with the signs reversed. I don't remember exactly, but it could have amounted to a upper-right quadrant versus lower-left quadrant graphical plotting of the vectors with essentially the same meaning. – Antoni Parellada May 30 '15 at 13:35
up vote 12 down vote accepted

1) The definition of eigenvector $Ax = \lambda x$ is ambidextrous. If $x$ is an eigenvector, so is $-x$, for then

$$A(-x) = -Ax = -\lambda x = \lambda (-x)$$

So the definition of an eigenbasis is ambiguous of sign.

2) It's hard to know for sure, but I have a strong suspicion of what is happening here. Your equation

$$ (A - \lambda)x = 0 $$

is technically incorrect. The correct equation is

$$ (A - \lambda I)x$$

The first equation is often used as a shorthand for the second. In general, this is unambiguous, because there is no real mathematical way to subtract a vector from a square matrix, but it is abuse of notation. In R though, you have broadcasting. So if you do

> M <- matrix(c(1, 1, 1, 1), nrow=2)
> M - .5
     [,1] [,2]
[1,]  0.5  0.5
[2,]  0.5  0.5

its not really what you want. The proper way would be

> M - diag(.5, 2)
     [,1] [,2]
[1,]  0.5  1.0
[2,]  1.0  0.5

The reason you are getting zero solutions is that the matrix you are starting with $A$ is invertible. More than likely (almost surely), the matrix you get by subtracting the same number from every entry will also be invertible. For invertible matrices, the only solution to $Ax = 0$ is the zero vector.

  • Thanks @ Matthew Drry . You're right my equation is used as a shorthand for the second. It's true that if A is invertible, Ax=0 can be written as x=0*A(-1).So x is always zero vector. As I mentioned,the condition that characterizes an eigenvalue, λ, is the existence of a nonzero vector x such that Ax=λx. How R can find a nonzero vector x?Because when i took eigen function, I got nonzero eigenvectors.Does it mean that behind eigen function, R find eigenvectors by another way which is not the solve function? – Kroll DU May 31 '15 at 10:14
  • 2
    @KrollDU You won't be able to use solve to recover the eigenvectors, because solve is designed for non-singular systems, and the eigenvector system is, by design, singular. So no, R does not use solve internally in it's eigen computation. You can check this by looking at the source code for eigen, where you'll find z <- if (!complex.x).Internal(La_rs(x, only.values)) – Matthew Drury May 31 '15 at 14:41
  • Looks like La_rs is a C function calling into a fortran routine, you can find it here: github.com/wch/r-source/blob/… – Matthew Drury May 31 '15 at 14:42
  • 1
    @ Matthew Drury, You're right. Solve function can't be used to recover the eigenvectors. Thanks so much for your help. – Kroll DU May 31 '15 at 14:57
  • @ Mathew if the definition of an eigenbasis is ambiguous of sign, what's the impact to the final result? – Kroll DU May 31 '15 at 15:01

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.