6
$\begingroup$

If I center my variables and then run a PCA, do I need to interpret negative eigenvectors different than positive eigenvectors?

Clarification: In my PCA I have in a component both negative and positive variables. This is probably very basic, but I've been told different things about the interpretation of the variables within a component, so I just need some clarification. Is it so that the negative values within a component moves in the same direction and the positive in the opposite? Or is it so that I should only look at the absolute values of the variables within a component?

Improve this question
$\endgroup$
5
  • 7
    $\begingroup$ What does it mean that a vector is negative? If $\mathbf{x}$ is an eigenvector of the matrix $\mathbf{A}$ then $\mathbf{Ax}=\lambda\mathbf{x}$ for some $\lambda$. But then $A(-\mathbf{x})=-A\mathbf{x}=\lambda(-\mathbf{x})$ so $-\mathbf{x}$ is also an eigenvector. $\endgroup$
    – MånsT
    Commented Apr 12, 2012 at 12:08
  • 3
    $\begingroup$ Hanne, welcome to our site! I hope you won't mind if I suggest your question indicates you would benefit from reading some explanations of PCA, such as the ones found at stats.stackexchange.com/questions/2691. In that thread I offer an explanation of the eigenvectors as principal directions of an ellipsoid: this provides an immediate geometric demonstration of the comment by MånsT, because whenever $\mathbf{x}$ is a principal direction, then $-\mathbf{x}$ is also principal due to the central symmetry of all ellipsoids. It is arbitrary which vector you pick to represent that direction. $\endgroup$
    – whuber ♦
    Commented Apr 12, 2012 at 14:36
  • 1
    $\begingroup$ I don't mind, thank you @whuber. I think I need to rephrase my question. In my PCA analysis I have in a component both negative and positive variables. This is probably very basic, but I've been told different things about the interpretation of the variables within a component, so I just need some clarification. First; Is it so that the negative values within a component moves in the same direction and the positive in the opposite? Or is it so that I should only look at the absolute values of the variables within a component? I hope this makes sense. $\endgroup$
    – Hanne
    Commented Apr 12, 2012 at 22:30
  • 1
    $\begingroup$ Ah, so what you mean is how to interpret the relative signs of the coefficients of the eigenvectors. (Even the signs are meaningless because negating an eigenvector will reverse all the signs, but the relative signs do not change, no matter how the eigenvector is scaled, and therefore are meaningful.) Because not everybody reads all comments, it would be good for you to edit your question to make this clear. $\endgroup$
    – whuber ♦
    Commented Apr 13, 2012 at 13:33
  • 1
    $\begingroup$ For more on the signs and how they might be managed, please see stats.stackexchange.com/questions/34396/…. $\endgroup$
    – whuber ♦
    Commented Aug 27, 2012 at 18:06

3 Answers 3

Reset to default
4
$\begingroup$

I think you have it backwards. If the value is positive, then a higher score on that variable is associated with a higher score on the component, if the value is negative, then a higher score implies a lower score on the component.

In addition, people sometimes use PCA to determine whether to keep or combine certain variables for a subsequent analysis. This is not, strictly speaking, an appropriate use of PCA. Factor analysis should be used for this purpose, but at any rate, people do it. In such a case, people will look at the absolute value to see if it is above some arbitrary threshold, such as .5, and if so, retain (or combine), and if not, drop. For what it's worth, I don't recommend this.

Update: I can't tell if I answered the right question or not. @whuber's second comment, in my opinion, is right on the money, and also consistent with my first paragraph above. However, the question is now different than before, and different from how I understand @whuber's comment, so I am a little confused. Essentially, PCA solves for the eigenvectors and eigenvalues. Neither will be negative whether or not you centered your variables first. The eigenvalues are the lengths of the corresponding eigenvectors. Just as I cannot buy a board -10 feet (i.e., -3 meters) long to build a patio, you cannot have a negative eigenvalue. The eigenvector returned will also be positive. You could negate it by multiplying all the signs by -1, but as @whuber notes, that would be meaningless. Once again as @whuber notes, the relative signs are meaningful, and their relation to the component is as I stated in my first paragraph above. That is, the relative signs (negative vs. positive) will denote the same relationship between higher (/ lower) scores on the variable and the component whether the variables were centered first or not.

Improve this answer
$\endgroup$
0
$\begingroup$

centering your variables shouldn't change the PCA results, as PCA first determines a correlation matrix and goes on from there. The correlations between your variables should be the same regardless, so the PCA results should not be affected by any mean centering you perform.

Improve this answer
$\endgroup$
6
  • 2
    $\begingroup$ I think in practice that is not always the case. Usually, $\mathbf X \mathbf X'$ is decomposed. If $\mathbf X$ is mean centered, $\mathbf X \mathbf X' = COV (\mathbf X)$, if it is mean centered and variance scaled, then $\mathbf X \mathbf X' = COR (\mathbf X)$. many (most) PCA functions will do the mean centering and/or variance scaling as part of their default pre-processing, but many also allow to switch this off. See e.g. the arguments center and scale. of prcomp in R. $\endgroup$
    – cbeleites
    Commented Apr 13, 2012 at 12:48
  • $\begingroup$ @cbeleites that is interesting, thanks. With regard to the original question, the centering should not change the PCA results, unless you specify alternative options when you call the PCA function? $\endgroup$
    – Luke
    Commented Apr 13, 2012 at 12:53
  • $\begingroup$ iff the default option is to center, then the results should not change (but they may flip as Måns explained). If you don't know whether your PCA algorithm does the centering and scaling, you can actually test it that way. $\endgroup$
    – cbeleites
    Commented Apr 13, 2012 at 13:14
  • 1
    $\begingroup$ Luke and @cb : Centering will hugely change the results in most cases: without centering, the center of the fitted ellipsoid must be at the origin, whereas with centering, the center of the ellipsoid will be near the center of the point cloud. It is geometrically obvious that these are entirely different things. $\endgroup$
    – whuber ♦
    Commented Apr 13, 2012 at 13:37
  • 2
    $\begingroup$ @whuber: Sure! Depending on the data, the uncentered first PC may actually be very close to the mean vector. However, most PCA functions will do the centering by default, and the results do not change whether the mean centering is done outside the PCA function or inside or both outside and again inside. And it may be quite hidden in the results, e.g. R's prcomp has the center but by default doesn't print it. I've also seen it reported it as the "$0^{th}$ component". So it may not be very obvious that it is actually done. $\endgroup$
    – cbeleites
    Commented Apr 13, 2012 at 14:28
0
$\begingroup$

When we say correlation that means can be two directional i.e. positive and negative. Interpretation of the principal components is based on finding which variables are most strongly correlated with each component, i.e., which of these numbers are large in magnitude, the farthest from zero in either positive or negative direction.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.