I am trying to extract the principal components from a dataset, but the eigenvectors and eigenvalues aren't aligned as I would expect them to be. Here's a simple example to illustrate.

import matplotlib.pyplot as plt
import numpy as np
x = [1, 2, 3, 4, 5]
y = [10, 7, 6, 4, 2]

Scatter plot

First I center the data, then calculate the covariance matrix, then extract the eigenvalues and eigenvectors.

centered_x = x - np.mean(x)
centered_y = y - np.mean(y)
X = np.array(list(zip(centered_x, centered_y)))
cov_mat = np.cov(X.T)

e_vals, e_vecs = np.linalg.eig(cov_mat)

Here is the result I get.

[[-0.88779092  0.46024699]
 [-0.46024699 -0.88779092]]
[ 0.03751344 11.66248656]

This isn't what I was expecting so I think here's where my misunderstanding comes in. I am interpreting this as I have two principal components, -0.88779092 0.46024699 with a weight of the square root of 0.03751344, and a second one of -0.46024699 -0.88779092 a weight of the square root of 11.66248656. Although the vectors seem right to me, the scaling seems flipped. The first principal component should be -0.88779092 0.46024699 with a weight of 11.66248656. So why are they flipped? What am I missing?

  • $\begingroup$ I hate to say it, but experience teaches that detailed interpretation of a PCA is a matter of ROTFM: what does the documentation say about the meaning of the output? Are the eigenvectors the columns or the rows of the matrix?? $\endgroup$ – whuber Mar 13 '19 at 17:55

I don't think the answer is wrong. The variance explained by vector -0.4 & 0.8 is the maximum. You are interpreting the results wrongly. If you read the documentation each column corresponds to 1 eigenvector. To prove your correctness I tried doing the same thing with scikit-learn PCA.

from sklearn.decomposition import PCA
import numpy as np
p = PCA()
x = [1, 2, 3, 4, 5]
y = [10, 7, 6, 4, 2]
x = x-np.mean(x)
y = y-np.mean(y)
X = np.array(list(zip(x, y)))
ind = 0
for eigen_vec in p.components_:
    print("Eigenvector : ",eigen_vec)
    print("Variance Explained : ",p.explained_variance_[ind])
    print("Ratio of Variance Explained : ",p.explained_variance_[ind])    

The output obtained was the following:-

Eigenvector :  [-0.46024699  0.88779092]
Variance Explained :  11.662486559124249
Ratio of Variance Explained :  0.9967937230020721
Eigenvector :  [-0.88779092 -0.46024699]
Variance Explained :  0.03751344087575544
Ratio of Variance Explained :  0.0032062769979278143
  • $\begingroup$ That's the result I was expecting. The eigenvector with the most variance explained goes up and to the left. But my eigenvector corresponding to the largest eigenvalues goes DOWN and to the left. $\endgroup$ – jss367 Mar 13 '19 at 17:37
  • $\begingroup$ You're saying vector -0.4 & 0.8 is the maximum (which I agree with) but that one has a lesser eigenvalue associated with it. $\endgroup$ – jss367 Mar 13 '19 at 17:53
  • $\begingroup$ @jss367 Edited my answer, you just interpreted the results wrongly. $\endgroup$ – Axelius Mar 13 '19 at 17:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.