# Principal component weights flipped after PCA

I am trying to extract the principal components from a dataset, but the eigenvectors and eigenvalues aren't aligned as I would expect them to be. Here's a simple example to illustrate.

import matplotlib.pyplot as plt
import numpy as np
x = [1, 2, 3, 4, 5]
y = [10, 7, 6, 4, 2]
plt.scatter(x,y); First I center the data, then calculate the covariance matrix, then extract the eigenvalues and eigenvectors.

centered_x = x - np.mean(x)
centered_y = y - np.mean(y)
X = np.array(list(zip(centered_x, centered_y)))
cov_mat = np.cov(X.T)

e_vals, e_vecs = np.linalg.eig(cov_mat)
print(e_vecs)
print(e_vals)


Here is the result I get.

[[-0.88779092  0.46024699]
[-0.46024699 -0.88779092]]
[ 0.03751344 11.66248656]


This isn't what I was expecting so I think here's where my misunderstanding comes in. I am interpreting this as I have two principal components, -0.88779092 0.46024699 with a weight of the square root of 0.03751344, and a second one of -0.46024699 -0.88779092 a weight of the square root of 11.66248656. Although the vectors seem right to me, the scaling seems flipped. The first principal component should be -0.88779092 0.46024699 with a weight of 11.66248656. So why are they flipped? What am I missing?

• I hate to say it, but experience teaches that detailed interpretation of a PCA is a matter of ROTFM: what does the documentation say about the meaning of the output? Are the eigenvectors the columns or the rows of the matrix?? – whuber Mar 13 '19 at 17:55

I don't think the answer is wrong. The variance explained by vector -0.4 & 0.8 is the maximum. You are interpreting the results wrongly. If you read the documentation each column corresponds to 1 eigenvector. To prove your correctness I tried doing the same thing with scikit-learn PCA.

from sklearn.decomposition import PCA
import numpy as np
p = PCA()
x = [1, 2, 3, 4, 5]
y = [10, 7, 6, 4, 2]
x = x-np.mean(x)
y = y-np.mean(y)
X = np.array(list(zip(x, y)))
p.fit(X)
ind = 0
for eigen_vec in p.components_:
print("Eigenvector : ",eigen_vec)
print("Variance Explained : ",p.explained_variance_[ind])
print("Ratio of Variance Explained : ",p.explained_variance_[ind])
ind=ind+1


The output obtained was the following:-

Eigenvector :  [-0.46024699  0.88779092]
Variance Explained :  11.662486559124249
Ratio of Variance Explained :  0.9967937230020721
Eigenvector :  [-0.88779092 -0.46024699]
Variance Explained :  0.03751344087575544
Ratio of Variance Explained :  0.0032062769979278143

• That's the result I was expecting. The eigenvector with the most variance explained goes up and to the left. But my eigenvector corresponding to the largest eigenvalues goes DOWN and to the left. – jss367 Mar 13 '19 at 17:37
• You're saying vector -0.4 & 0.8 is the maximum (which I agree with) but that one has a lesser eigenvalue associated with it. – jss367 Mar 13 '19 at 17:53
• @jss367 Edited my answer, you just interpreted the results wrongly. – Axelius Mar 13 '19 at 17:56