# Confused about the visual explanation of eigenvectors: how can visually different datasets have the same eigenvectors?

A lot of statistics textbooks provide an intuitive illustration of what the eigenvectors of a covariance matrix are: The vectors u and z form the eigenvectors (well, eigenaxes). This makes sense. But the one thing that confuses me is that we extract eigenvectors from the correlation matrix, not the raw data. Furthermore, raw datasets that are quite different can have identical correlation matrices. For example, the following both have correlation matrices of:

$$\left[\begin{array}{} 1 & 0.97 \\ 0.97 &1\end{array}\right]$$ As such they have eigenvectors pointing in the same direction:

$$\left[\begin{array}{} .71 & -.71 \\ .71 & .71\end{array}\right]$$

But if you were to apply the same visual interpretation of which directions the eigenvectors were in the raw data, you would get vectors pointing in different directions.

Can someone please tell me where I have gone wrong?

Second Edit: If I may be so bold, with the excellent answers below I was able to make sense of the confusion and have illustrated it.

1. The visual explanation coheres with the fact that the eigenvectors extracted from the covariance matrix are distinct.

Covariances and Eigenvectors (Red):

$$\left[\begin{array}{} 1 & 1 \\ 1 & 1\end{array}\right] \left[\begin{array}{} .7 & -.72 \\ .72 & .7\end{array}\right]$$

Covariances and Eigenvectors (Blue):

$$\left[\begin{array}{} .25 & .5 \\ .5 & 1\end{array}\right] \left[\begin{array}{} .43 & -.9 \\ .9 & .43\end{array}\right]$$

2. Correlation matrices reflect the covariance matrices of the standardized variables. Visual inspection of the standardized variables demonstrate why identical eigenvectors are extracted in my example: • If you wish to evaluate correlation, then you must draw your scatterplots with scales in which the standard deviations of the components are equal. That is not the case in any of your images (except perhaps for the red dots in the second one), which may be one reason you find this confusing. – whuber Nov 17 '14 at 21:24
• I appreciate your having illustrated your question. That helps people understand it & adds to the value of the thread for future reference. Be aware, however, that ~10% of men are red-green colorblind. With 2 colors, red & blue may be safer. – gung - Reinstate Monica Nov 17 '14 at 21:42
• Many thanks, I have corrected the colors as you suggested – Sue Doh Nimh Nov 17 '14 at 22:25
• No problem, @SueDohNimh. Thank you for making it intelligible for all. On a different note, I would keep the [PCA] tag. If you want to re-focus the question, or ask a new (related) question & link to this one, that seems fine, but I think this question is PCA-ish enough to merit the tag. – gung - Reinstate Monica Nov 17 '14 at 22:36
• Nice job, @SueDohNimh. You could also add that as an answer to your own question instead of an edit, if you wanted to. – gung - Reinstate Monica Nov 18 '14 at 0:05

In your second figure, the correlations are the same, but the groups look different. They look different because they have different covariances. However, the variances are also different (e.g., the red group varies over a wider range of X1), and the correlation is the covariance divided by the standard deviations (${\rm Cov}_{xy} / {\rm SD}_x{\rm SD}_y$). As a result, the correlations can be the same.
• +1 You have probably also noticed that with two variables the correlation matrix always has the same two eigenvectors, $(1,1)$ and $(1,-1)$, no matter what value the correlation has. – whuber Nov 17 '14 at 21:33