1
$\begingroup$

I'm struggling on getting a good explanation for the unexpected signs of Principal Component for months. I tried to replicate a result and I got exactly the opposite signs for all components. While I searched through forums for an answer, I found this website I'm getting "jumpy" loadings in rollapply PCA in R. Can I fix it?

From what I have read from this website, I understand that we can reverse the sign of components based on other criteria - e.g. EURO trend - as what have been mentioned by @amoeba. I'm wondering if there is a book or academic literature that says that we can flip the sign in this way (i.e. based on external factor)? I need a strong support for my research paper if I flip the signs of all components in this way. Hence, I would greatly appreciate if someone can recommend me some books that talk about this issue?

And also @amoeba mentioned that the signs are consistent in sliding PCA. Does it mean that we should have the same combination for each window (for example +a, +b, -c in first window & -a, -b, +c in second window)? So, if I think the signs in second window are correct, then I will flip the 1st window's vector and both vectors will have the same sign by then. What if they have different combination (e.g. +a, -b, -c in second window)? I think their correlation could have changed from time to time and hence we will have different combinations in different windows?

| cite | improve this question | |
$\endgroup$
  • 6
    $\begingroup$ It's unclear what needs a "proof." Since the sign of an eigenvector is arbitrary, you can use any procedure you like to set it. Could you show us specifically how your question differs from the one you cite? $\endgroup$ – whuber♦ Mar 23 '17 at 19:24
  • 3
    $\begingroup$ Regarding the sign being arbitrary see also stats.stackexchange.com/questions/88880. You don't need any "book or academic literature" allowing you to reverse the sign; you can simply reverse it as you like. $\endgroup$ – amoeba Mar 23 '17 at 19:25
  • $\begingroup$ @whuber I have the same question with previous posts and I have better understanding now on the flipping sign. But, I would like to ask if there is any literature can be used as a support to justify why I flip the sign based on some external factor. Amoeba has just answered the question. Reg. my second question, I'm wondering if the signs of all components should be the same for every window? What if I have different combination of sign in every window (e.g. +a, +b, -c in my 1st window and -a, +b, +c in my 2nd window)? $\endgroup$ – SNU Mar 23 '17 at 20:10
5
$\begingroup$

Something that various people are pointing out is that the vectors $(1,1)$, $(2,2)$, or $(-1,-1)$ all represent the same line. When you find an eigenvector, what's uniquely determined is the line, not the actual vector.

An eigenvector for a matrix (linear transformation) $A$ is defined as any vector $\mathbf{v} \neq \mathbf{0}$ which satisfies:

$$A \mathbf{v} = \lambda \mathbf{v}$$

If $\mathbf{v}$ is an eigenvector, any scalar multiple $\hat{\mathbf{v}} = \alpha \mathbf{v}$ will also work ($\alpha \neq 0$):

\begin{align*} A \mathbf{v} = \lambda \mathbf{v}\quad & \Leftrightarrow \quad \alpha A \mathbf{v} = \alpha \lambda \mathbf{v} \\ & \Leftrightarrow \quad A\hat{\mathbf{v}} = \lambda \hat{\mathbf{v}} \end{align*}

Eg. Choose $\alpha = -1$. If $\mathbf{v}$ is an eigenvector, so is $-\mathbf{v}$.

Let's say your PCA algorithm guarantees you that $\|\mathbf{v}\| = 1$. You still have two possibilities because if you take the intersection of a line through the origin and the unit circle, you get two points.

enter image description here

In this example, whether you have $\mathbf{v} = (\frac{\sqrt{2}}{2},\frac{\sqrt{2}}{2})$ or $\mathbf{v} = (-\frac{\sqrt{2}}{2},-\frac{\sqrt{2}}{2})$, it really doesn't matter.

A fun example of flipping the sign on basis vectors: upside down map

Instead of a map where the y-axis measures how far north and the x-axis measures how far east, you could just as easily have a map where the y-axis measures how far south and the x-axis measures how far west.

Up and down would still be aligned with the magnetic axis.

alt text

| cite | improve this answer | |
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.