I am trying to understand PCA implemented in different methods on python. I am failing to get equal PCA coefficients in each of the methods. By PCA coefficients I mean data projected in the principle components space. Note that I did sort the eigen values and vectors of the COV matrix.
In the code below, I am expecting to get the same coeffecients Z1, Z2, Z3 regardless of the method used. However, I am not.
import numpy as np
from sklearn.decomposition import PCA
X=np.array([[3,2,1],[2,4,5],[1,2,3],[0,2,5]])
xm=np.mean(X,axis=0)
print('Sample mean:')
print(xm)
Xs=X-xm
Q=np.cov(Xs)
print('Cov matrix:')
print(Q)
eigVals, eigVec = np.linalg.eig(Q)
idx = np.argsort(eigVals)[::-1]
eigVec = eigVec[:,idx]
eigVals = eigVals[idx]
print('Eig Values:')
print(eigVals)
print('Eig Vectors:')
print(eigVec)
Z1=np.dot(np.transpose(eigVec),Xs)
print('Coeff using covariance matrix are: ')
print(Z1)
U, s, Vtr= np.linalg.svd(Xs, full_matrices = False)
print('Coeff using SVD are: ')
Z2=U*s**2
print(Z2)
pca = PCA()
Z3=pca.fit_transform(X)
print('Coeff using pca command are: ')
print(Z3)
This is the output of the code:
Sample mean:
[1.5 2.5 3.5]
Cov matrix:
[[ 4. -1. 0. -3. ]
[-1. 0.33333333 0. 0.66666667]
[ 0. 0. 0. 0. ]
[-3. 0.66666667 0. 2.33333333]]
Eig Values:
[6.51313067e+00 1.53535995e-01 2.54467750e-16 0.00000000e+00]
Eig Vectors:
[[-0.78286395 0.23192824 0.57735027 0. ]
[ 0.19057622 -0.79394419 0.57735027 0. ]
[ 0. 0. 0. 1. ]
[ 0.59228772 0.56201594 0.57735027 0. ]]
Coeff using covariance matrix are:
[[-1.96743939 0.38115244 3.13145578]
[-0.89210364 -1.58788837 -0.92771297]
[ 0.28867513 0.28867513 0.28867513]
[-0.5 -0.5 -0.5 ]]
Coeff using SVD are:
[[-1.11401957e+01 -3.81533466e-01 2.17291577e-02]
[ 5.17496857e+00 -3.67010581e+00 -4.86275118e-03]
[-1.15810216e+00 1.70487869e+00 -4.67765294e-02]
[ 7.12332929e+00 2.34676059e+00 2.99101229e-02]]
Coeff using pca command are:
[[ 2.95145599 0.17610969 -0.0888421 ]
[-1.37104342 1.69406159 0.0198819 ]
[ 0.30682473 -0.78694448 0.19125108]
[-1.8872373 -1.0832268 -0.12229089]]