Reverse the sign of PCA [duplicate]

Question

I'm struggling on getting a good explanation for the unexpected signs of Principal Component for months. I tried to replicate a result and I got exactly the opposite signs for all components. While I searched through forums for an answer, I found this website I'm getting "jumpy" loadings in rollapply PCA in R. Can I fix it?

From what I have read from this website, I understand that we can reverse the sign of components based on other criteria - e.g. EURO trend - as what have been mentioned by @amoeba. I'm wondering if there is a book or academic literature that says that we can flip the sign in this way (i.e. based on external factor)? I need a strong support for my research paper if I flip the signs of all components in this way. Hence, I would greatly appreciate if someone can recommend me some books that talk about this issue?

And also @amoeba mentioned that the signs are consistent in sliding PCA. Does it mean that we should have the same combination for each window (for example +a, +b, -c in first window & -a, -b, +c in second window)? So, if I think the signs in second window are correct, then I will flip the 1st window's vector and both vectors will have the same sign by then. What if they have different combination (e.g. +a, -b, -c in second window)? I think their correlation could have changed from time to time and hence we will have different combinations in different windows?

Answer 1

Something that various people are pointing out is that the vectors $(1,1)$, $(2,2)$, or $(-1,-1)$ all represent the same line. When you find an eigenvector, what's uniquely determined is the line, not the actual vector.

An eigenvector for a matrix (linear transformation) $A$ is defined as any vector $\mathbf{v} \neq \mathbf{0}$ which satisfies:

$$A \mathbf{v} = \lambda \mathbf{v}$$

If $\mathbf{v}$ is an eigenvector, any scalar multiple $\hat{\mathbf{v}} = \alpha \mathbf{v}$ will also work ($\alpha \neq 0$):

\begin{align*} A \mathbf{v} = \lambda \mathbf{v}\quad & \Leftrightarrow \quad \alpha A \mathbf{v} = \alpha \lambda \mathbf{v} \\ & \Leftrightarrow \quad A\hat{\mathbf{v}} = \lambda \hat{\mathbf{v}} \end{align*}

Eg. Choose $\alpha = -1$. If $\mathbf{v}$ is an eigenvector, so is $-\mathbf{v}$.

Let's say your PCA algorithm guarantees you that $\|\mathbf{v}\| = 1$. You still have two possibilities because if you take the intersection of a line through the origin and the unit circle, you get two points.

In this example, whether you have $\mathbf{v} = (\frac{\sqrt{2}}{2},\frac{\sqrt{2}}{2})$ or $\mathbf{v} = (-\frac{\sqrt{2}}{2},-\frac{\sqrt{2}}{2})$, it really doesn't matter.

A fun example of flipping the sign on basis vectors: upside down map

Instead of a map where the y-axis measures how far north and the x-axis measures how far east, you could just as easily have a map where the y-axis measures how far south and the x-axis measures how far west.

Up and down would still be aligned with the magnetic axis.