4
$\begingroup$

If I center my variables and then run a PCA analysis, do I need to interpret negative eigenvectors different than positive eigenvectors?

Clarification: In my PCA analysis I have in a component both negative and positive variables. This is probably very basic, but I've been told different things about the interpretation of the variables within a component, so I just need some clarification. Is it so that the negative values within a component moves in the same direction and the positive in the opposite? Or is it so that I should only look at the absolute values of the variables within a component?

$\endgroup$
  • 5
    $\begingroup$ What does it mean that a vector is negative? If $\mathbf{x}$ is an eigenvector of the matrix $\mathbf{A}$ then $\mathbf{Ax}=\lambda\mathbf{x}$ for some $\lambda$. But then $A(-\mathbf{x})=-A\mathbf{x}=\lambda(-\mathbf{x})$ so $-\mathbf{x}$ is also an eigenvector. $\endgroup$ – MånsT Apr 12 '12 at 12:08
  • 2
    $\begingroup$ Hanne, welcome to our site! I hope you won't mind if I suggest your question indicates you would benefit from reading some explanations of PCA, such as the ones found at stats.stackexchange.com/questions/2691. In that thread I offer an explanation of the eigenvectors as principal directions of an ellipsoid: this provides an immediate geometric demonstration of the comment by MånsT, because whenever $\mathbf{x}$ is a principal direction, then $-\mathbf{x}$ is also principal due to the central symmetry of all ellipsoids. It is arbitrary which vector you pick to represent that direction. $\endgroup$ – whuber Apr 12 '12 at 14:36
  • 1
    $\begingroup$ I don't mind, thank you @whuber. I think I need to rephrase my question. In my PCA analysis I have in a component both negative and positive variables. This is probably very basic, but I've been told different things about the interpretation of the variables within a component, so I just need some clarification. First; Is it so that the negative values within a component moves in the same direction and the positive in the opposite? Or is it so that I should only look at the absolute values of the variables within a component? I hope this makes sense. $\endgroup$ – Hanne Apr 12 '12 at 22:30
  • $\begingroup$ Ah, so what you mean is how to interpret the relative signs of the coefficients of the eigenvectors. (Even the signs are meaningless because negating an eigenvector will reverse all the signs, but the relative signs do not change, no matter how the eigenvector is scaled, and therefore are meaningful.) Because not everybody reads all comments, it would be good for you to edit your question to make this clear. $\endgroup$ – whuber Apr 13 '12 at 13:33
  • $\begingroup$ For more on the signs and how they might be managed, please see stats.stackexchange.com/questions/34396/…. $\endgroup$ – whuber Aug 27 '12 at 18:06
3
$\begingroup$

I think you have it backwards. If the value is positive, then a higher score on that variable is associated with a higher score on the component, if the value is negative, then a higher score implies a lower score on the component.

In addition, people sometimes use PCA to determine whether to keep or combine certain variables for a subsequent analysis. This is not, strictly speaking, an appropriate use of PCA. Factor analysis should be used for this purpose, but at any rate, people do it. In such a case, people will look at the absolute value to see if it is above some arbitrary threshold, such as .5, and if so, retain (or combine), and if not, drop. For what it's worth, I don't recommend this.

Update: I can't tell if I answered the right question or not. @whuber's second comment, in my opinion, is right on the money, and also consistent with my first paragraph above. However, the question is now different than before, and different from how I understand @whuber's comment, so I am a little confused. Essentially, PCA solves for the eigenvectors and eigenvalues. Neither will be negative whether or not you centered your variables first. The eigenvalues are the lengths of the corresponding eigenvectors. Just as I cannot buy a board -10 feet (i.e., -3 meters) long to build a patio, you cannot have a negative eigenvalue. The eigenvector returned will also be positive. You could negate it by multiplying all the signs by -1, but as @whuber notes, that would be meaningless. Once again as @whuber notes, the relative signs are meaningful, and their relation to the component is as I stated in my first paragraph above. That is, the relative signs (negative vs. positive) will denote the same relationship between higher (/ lower) scores on the variable and the component whether the variables were centered first or not.

$\endgroup$
0
$\begingroup$

centering your variables shouldn't change the PCA results, as PCA first determines a correlation matrix and goes on from there. The correlations between your variables should be the same regardless, so the PCA results should not be affected by any mean centering you perform.

$\endgroup$
  • 1
    $\begingroup$ I think in practice that is not always the case. Usually, $\mathbf X \mathbf X'$ is decomposed. If $\mathbf X$ is mean centered, $\mathbf X \mathbf X' = COV (\mathbf X)$, if it is mean centered and variance scaled, then $\mathbf X \mathbf X' = COR (\mathbf X)$. many (most) PCA functions will do the mean centering and/or variance scaling as part of their default pre-processing, but many also allow to switch this off. See e.g. the arguments center and scale. of prcomp in R. $\endgroup$ – cbeleites supports Monica Apr 13 '12 at 12:48
  • $\begingroup$ @cbeleites that is interesting, thanks. With regard to the original question, the centering should not change the PCA results, unless you specify alternative options when you call the PCA function? $\endgroup$ – Luke Apr 13 '12 at 12:53
  • $\begingroup$ iff the default option is to center, then the results should not change (but they may flip as Måns explained). If you don't know whether your PCA algorithm does the centering and scaling, you can actually test it that way. $\endgroup$ – cbeleites supports Monica Apr 13 '12 at 13:14
  • $\begingroup$ To be precise: the above formulae are not exactly correct for $COV$ and $COR$ but are what PCA usually does. If you really want to calculate correlation or covariance, you need to divide by the degrees of freedom $COV(\mathbf X) = \frac{1}{df} \mathbf X_{centered} \mathbf X_{centered}^′$; e.g. for the sample covariance $df = n - 1$ unless you have extra knowledge like mean must be zero for theoretical reasons. $\endgroup$ – cbeleites supports Monica Apr 13 '12 at 13:23
  • 1
    $\begingroup$ @whuber: Sure! Depending on the data, the uncentered first PC may actually be very close to the mean vector. However, most PCA functions will do the centering by default, and the results do not change whether the mean centering is done outside the PCA function or inside or both outside and again inside. And it may be quite hidden in the results, e.g. R's prcomp has the center but by default doesn't print it. I've also seen it reported it as the "$0^{th}$ component". So it may not be very obvious that it is actually done. $\endgroup$ – cbeleites supports Monica Apr 13 '12 at 14:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.