8
$\begingroup$

Using PCA manually on correlation matrix, I'm getting imaginary numbers in both eigenvalues and eigenvectors. Is this expected behavior?

I understand that when interpreting a matrix as a linear transformation, the eigenvectors can be thought of as the "principal effects" of the transformation. The 'limit', one might say, is when the transformation is recursively applied to some arbitrary input vector for an infinite number of iterations and this vector is pulled into the span of the leading eigenvector (the other effects are simply washed out.)

I use this mental framework as it helps me make sense of rotational matrices. There is no principal effect; the arbitrary vector is rotated infinitely but never converges on a real span (hence the imaginary numbers.)

So I'm left wondering, if the output of PCA (using correlation matrix) has imaginary numbers in its eigenvectors and eigenvalues then it might be a rotational matrix - aka no span convergence. In this light, I'm doubtful that such eigenvectors can be of any use in some downstream task and simply removing the imaginary numbers sounds like a perilous choice.

Response to comments/answers:

I checked for symmetry using

def check_symmetric(a, rtol=1e-05, atol=1e-08):
    return np.allclose(a, a.T, rtol=rtol, atol=atol)

check_symmetric(correlation_matrix)

The response was true. Additionally, I used np.corrcoef for correlation matrix and numpy.linalg.eig for eigen decomposition.

Having verified these conditions, I wonder what else might be wrong...

One other side note is that the minimum correlation is 0.58, which seems a bit high, perhaps pointing to an issue with the input data? (The max correlation is of course 1.0)

$\endgroup$
7
  • 1
    $\begingroup$ Did you call a function that assumes the correlation matrix is symmetric (e.g. numpy.linalg.eigh vs numpy.linalg.eig)? $\endgroup$
    – Sycorax
    Jan 22 at 19:15
  • $\begingroup$ I used the latter - numpy.linalg.eig but I believe that a correlation matrix should always be symmetric, no? $\endgroup$
    – jbuddy_13
    Jan 22 at 19:28
  • $\begingroup$ Likewise, used np.corrcoef for correlation matrix $\endgroup$
    – jbuddy_13
    Jan 22 at 19:32
  • 3
    $\begingroup$ eig will give imaginary outputs to large symmetric matrices because of numerical error. This is actually part of the reason why eigh exists. (in R, it is the same function eigen in either case, but set the argument "symmetric" to TRUE). $\endgroup$ Jan 22 at 19:52
  • 2
    $\begingroup$ Please provide a reproducible example -- or at least tell us what the eigenvalues are that you found! It sounds like you might just be either misreading the output or are focusing on an imaginary part that differs from zero by floating point roundoff. $\endgroup$
    – whuber
    Jan 22 at 20:27

1 Answer 1

7
$\begingroup$

A correlation matrix is symmetrical and hence all the eigenvalues are real.

I would verify a few things:

  • I would check if the correlation matrix is indeed a symmetric matrix.
  • Is the magnitude large for the imaginary part? If it is small, it is likely due to numerical issue.
$\endgroup$
3
  • $\begingroup$ Added some context to the question prompt, including, yes the matrix is symmetric. Additionally, the max Eigenvalue is 123.86+0j - which seems a bit large to your second question. @siong-thye-goh $\endgroup$
    – jbuddy_13
    Jan 22 at 19:52
  • 5
    $\begingroup$ @jbuddy_13, the maginutde of the imaginary part of your reported eigenvalue is numerically 0. $\endgroup$ Jan 22 at 19:53
  • 3
    $\begingroup$ @jbuddy_13 123.86+0j means 123.86 (a real number) plus 0j (zero imaginary number). So it's equivalent to 123.86 (a real number). So there is no imaginary number in that eigenvalue. $\endgroup$
    – justhalf
    Jan 23 at 4:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.