# Calculating PCA coefficients using SVD, PCA (sklearn) and Covariance Matrix

I am trying to understand PCA implemented in different methods on python. I am failing to get equal PCA coefficients in each of the methods. By PCA coefficients I mean data projected in the principle components space. Note that I did sort the eigen values and vectors of the COV matrix.

In the code below, I am expecting to get the same coeffecients Z1, Z2, Z3 regardless of the method used. However, I am not.

import numpy as np
from sklearn.decomposition import PCA

X=np.array([[3,2,1],[2,4,5],[1,2,3],[0,2,5]])
xm=np.mean(X,axis=0)
print('Sample mean:')
print(xm)
Xs=X-xm
Q=np.cov(Xs)
print('Cov matrix:')
print(Q)
eigVals, eigVec = np.linalg.eig(Q)

idx = np.argsort(eigVals)[::-1]
eigVec = eigVec[:,idx]
eigVals = eigVals[idx]

print('Eig Values:')
print(eigVals)

print('Eig Vectors:')
print(eigVec)
Z1=np.dot(np.transpose(eigVec),Xs)
print('Coeff using covariance matrix are: ')
print(Z1)

U, s, Vtr= np.linalg.svd(Xs, full_matrices = False)
print('Coeff using SVD are: ')
Z2=U*s**2
print(Z2)

pca = PCA()
Z3=pca.fit_transform(X)
print('Coeff using pca command are: ')
print(Z3)


This is the output of the code:

Sample mean:
[1.5 2.5 3.5]
Cov matrix:
[[ 4.         -1.          0.         -3.        ]
[-1.          0.33333333  0.          0.66666667]
[ 0.          0.          0.          0.        ]
[-3.          0.66666667  0.          2.33333333]]
Eig Values:
[6.51313067e+00 1.53535995e-01 2.54467750e-16 0.00000000e+00]
Eig Vectors:
[[-0.78286395  0.23192824  0.57735027  0.        ]
[ 0.19057622 -0.79394419  0.57735027  0.        ]
[ 0.          0.          0.          1.        ]
[ 0.59228772  0.56201594  0.57735027  0.        ]]
Coeff using covariance matrix are:
[[-1.96743939  0.38115244  3.13145578]
[-0.89210364 -1.58788837 -0.92771297]
[ 0.28867513  0.28867513  0.28867513]
[-0.5        -0.5        -0.5       ]]
Coeff using SVD are:
[[-1.11401957e+01 -3.81533466e-01  2.17291577e-02]
[ 5.17496857e+00 -3.67010581e+00 -4.86275118e-03]
[-1.15810216e+00  1.70487869e+00 -4.67765294e-02]
[ 7.12332929e+00  2.34676059e+00  2.99101229e-02]]
Coeff using pca command are:
[[ 2.95145599  0.17610969 -0.0888421 ]
[-1.37104342  1.69406159  0.0198819 ]
[ 0.30682473 -0.78694448  0.19125108]
[-1.8872373  -1.0832268  -0.12229089]]

• What is different? What is your expected behavior, and how does the observed behavior differ?
– Sycorax
Commented Apr 29, 2019 at 20:41
• I edited the question to explain this part - also added the output of the code Commented Apr 29, 2019 at 20:44