I have been studying how to interpret principal components.

I recently came across an example of a particular eigenvector:

$$e_j^T = \left[ \frac{\sqrt{2} }{2}, \frac{-\sqrt{2} }{2}, 0, \dots,0 \right]$$

I am under the impression that, often in social sciences, "this is called a contrast."

From what I can see, we have the difference between $X_1$ and $X_2$ (variables in our original dataset) playing a role in the $j$th eigenvector.

What can this say about the relationship between $X_1$ and $X_2$, with respect to this eigenvector? There is clearly something going on with the equal, yet opposite, contribution of these independent variables, to our (possibly first) principal component. I guess the answer to this is the answer to the question, "what is a contrast?"

Strangely enough, googling hasn't coming up with much of anything aside from dense discussion such as found on the Wikipedia page.

  • 1
    $\begingroup$ Some corrections are needed to stave of possible confusion: first, $e_j^\prime$ is an eigenvector, not an eigenvalue. Second, it does not assert any relationship at all between $X_1$ and $X_2$: it is literally just a linear combination of the two. FWIW, googling "linear regression test contrast" turns up the Wikipedia article on the subject right away. $\endgroup$ – whuber Mar 29 '14 at 20:54
  • 1
    $\begingroup$ argh, thanks, I've fixed my typos and generally cleaned up the question. $\endgroup$ – Matt O'Brien Mar 29 '14 at 21:34
  • 1
    $\begingroup$ I'm confused...I was taught it's dangerous to interpret PCA components because they are not identifiable, only the subspace they span is identifiable. $\endgroup$ – anonymous Jun 5 '15 at 18:49
  • $\begingroup$ @user43228 That's right. I think it's understood that $e_j^T$ is being considered only up to a nonzero multiple. The usual way to express it as a contrast would be as the vector $(1,-1,0,\ldots,0)$ or $(-1,1,0,\ldots,0)$. That is why Matt (the OP) is using terms like "equal, yet opposite contribution" without focusing on the magnitudes of the components. $\endgroup$ – whuber Jun 5 '15 at 18:56
  • 1
    $\begingroup$ @aabeshou On the contrary, one can interpret individual PCA components (and this is often done). Nevertheless, I agree with your question and answer. $\endgroup$ – whuber Jun 5 '15 at 20:48

Remembering that interpreting PCA results can be an art-form, let's use a relatively simple data set. The link here shows a PCA and interpretation of Fisher's iris data, which you can easily obtain for yourself and analyse in the software of your choice.

The bottom of p.9 shows the eigenvalues for the four principal components and p.11 provides an interpretation of the first two principal components.

As you can see, PC2 has been called a contrast between sepal length/width and petal length/width. The reason for that is that the eigenvector values corresponding to the sepal measurements are all negative and those corresponding to the petal measurements are all positive. However, and this is where the PCA "art" interpretation comes into it, you could argue that petal width has little effect on PC2, as it is only 0.08 and therefore not a very large value.

Looking at PC3, and ignoring the fact it accounts for such little variance (as that is not the point of this discussion), it appears to be a contrast between sepal and petal width, on the one hand, and sepal length on the other.

Having a subject matter expert on hand to help interpret PCA results can be very useful.

  • $\begingroup$ Hi gung, just been busy with study and work. Thanks for the hello. :) $\endgroup$ – Michelle Mar 30 '14 at 8:59
  • $\begingroup$ +1 and I took the liberty of correcting what seemed a mistake/typo in one sentence, confusing eigenvalues and eigenvectors. Please check that this is indeed what you had in mind. $\endgroup$ – amoeba says Reinstate Monica Dec 29 '14 at 13:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.