0
$\begingroup$

I am comparing the output from the singular value decomposition with the eigendecomposition of the covariance matrix (symmetric matrix). I am expecting that the Eigenvector and a non-diagonal matrix of SVD to be similar, yet they are not. I found that I need to flip and multip one column by -1. Pricnc[pe components could be calculated using either the eigendecomposition or SVD. Given that the eigendecomposition give different output from the SVD for the covariance matrix may yield different PCA output.

import numpy as np
import matplotlib.pyplot as plt 

A=np.array([[1,3,2],
       [4,8,2],
       [3,9,7],
       [22,11,17],
       [55,33,66]])


A=A-np.mean(A,0)
print(np.dot(A.T,A)/(A.shape[0]-1))
co=np.cov(A.T)
print(co)
#%%
[D,UI]=np.linalg.eigh(co)
[aa,bb,cc]=np.linalg.svd(co)
print (UI)
print(aa)

UI
array([[-0.15348562, -0.77609875, -0.61164768],
   [-0.85628748,  0.41338399, -0.30965372],
   [ 0.49316722,  0.47621886, -0.72801215]])

aa
array([[-0.61164768,  0.77609875, -0.15348562],
   [-0.30965372, -0.41338399, -0.85628748],
   [-0.72801215, -0.47621886,  0.49316722]])

aa[:,::-1]
array([[-0.15348562,  0.77609875, -0.61164768],
   [-0.85628748, -0.41338399, -0.30965372],
   [ 0.49316722, -0.47621886, -0.72801215]])
$\endgroup$

marked as duplicate by Sycorax, whuber Mar 21 at 13:46

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

2
$\begingroup$

From the definition of an eigenvector $v$ and eigenvalue $\lambda$ $$ \begin{align}Av &=\lambda v \\ A(-v) &= \lambda (-v) \end{align}$$ we can see that eigenvectors are only identified up to a nonzero multiple. Your discovery that multiplying by $-1$ makes the results coincide is not surprising, and amounts to a difference in how the two methods work.

$\endgroup$
  • $\begingroup$ thank, any idea how to explain mismatch in only part of the matrix (one column) and not the whole matrix. I added the values of the matrix that I get above. Thanks $\endgroup$ – Ahmed Mar 20 at 23:52
  • 1
    $\begingroup$ @kernel One column is an eigenvector. My answer says that the sign and magnitude of an eigenvector is arbitrary. $\endgroup$ – Sycorax Mar 20 at 23:56

Not the answer you're looking for? Browse other questions tagged or ask your own question.