1
$\begingroup$

I am trying to understand PCA implemented in different methods on python. I am failing to get equal PCA coefficients in each of the methods. By PCA coefficients I mean data projected in the principle components space. Note that I did sort the eigen values and vectors of the COV matrix.

In the code below, I am expecting to get the same coeffecients Z1, Z2, Z3 regardless of the method used. However, I am not.

import numpy as np
from sklearn.decomposition import PCA

X=np.array([[3,2,1],[2,4,5],[1,2,3],[0,2,5]])
xm=np.mean(X,axis=0)
print('Sample mean:')
print(xm)
Xs=X-xm
Q=np.cov(Xs)
print('Cov matrix:')
print(Q)
eigVals, eigVec = np.linalg.eig(Q)

idx = np.argsort(eigVals)[::-1]
eigVec = eigVec[:,idx]
eigVals = eigVals[idx]

print('Eig Values:')
print(eigVals)

print('Eig Vectors:')
print(eigVec)
Z1=np.dot(np.transpose(eigVec),Xs)
print('Coeff using covariance matrix are: ')
print(Z1) 

U, s, Vtr= np.linalg.svd(Xs, full_matrices = False)
print('Coeff using SVD are: ')
Z2=U*s**2
print(Z2)

pca = PCA()
Z3=pca.fit_transform(X)
print('Coeff using pca command are: ')
print(Z3)

This is the output of the code:

Sample mean:
[1.5 2.5 3.5]
Cov matrix:
[[ 4.         -1.          0.         -3.        ]
 [-1.          0.33333333  0.          0.66666667]
 [ 0.          0.          0.          0.        ]
 [-3.          0.66666667  0.          2.33333333]]
Eig Values:
[6.51313067e+00 1.53535995e-01 2.54467750e-16 0.00000000e+00]
Eig Vectors:
[[-0.78286395  0.23192824  0.57735027  0.        ]
 [ 0.19057622 -0.79394419  0.57735027  0.        ]
 [ 0.          0.          0.          1.        ]
 [ 0.59228772  0.56201594  0.57735027  0.        ]]
Coeff using covariance matrix are: 
[[-1.96743939  0.38115244  3.13145578]
 [-0.89210364 -1.58788837 -0.92771297]
 [ 0.28867513  0.28867513  0.28867513]
 [-0.5        -0.5        -0.5       ]]
Coeff using SVD are: 
[[-1.11401957e+01 -3.81533466e-01  2.17291577e-02]
 [ 5.17496857e+00 -3.67010581e+00 -4.86275118e-03]
 [-1.15810216e+00  1.70487869e+00 -4.67765294e-02]
 [ 7.12332929e+00  2.34676059e+00  2.99101229e-02]]
Coeff using pca command are: 
[[ 2.95145599  0.17610969 -0.0888421 ]
 [-1.37104342  1.69406159  0.0198819 ]
 [ 0.30682473 -0.78694448  0.19125108]
 [-1.8872373  -1.0832268  -0.12229089]]
$\endgroup$
2
  • 1
    $\begingroup$ What is different? What is your expected behavior, and how does the observed behavior differ? $\endgroup$
    – Sycorax
    Commented Apr 29, 2019 at 20:41
  • $\begingroup$ I edited the question to explain this part - also added the output of the code $\endgroup$
    – HaneenSu
    Commented Apr 29, 2019 at 20:44

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.