1
$\begingroup$

In PCA and LDA techniques, eigenvectors with the $k$ largest eigenvalues give principal components. However, when selecting these eigenvalues, are they to be sorted by the absolute value (regardless of sign) of the eigenvalues or just the eigenvalues themselves (with sign) ?

That is, are the eigenvectors decided by the order of dominant eigenvalues? And are thus just the dominant eigenvectors? [Dominant eigenvalues as defined here]

If yes, can you provide a simple intuitive explanation of why sign of eigenvalue does not matter.

I found some implementations doing so.

Improve this question
$\endgroup$
6
  • 3
    $\begingroup$ The covariance matrix used for PCA does not have negative eigenvalues! $\endgroup$
    – kjetil b halvorsen ♦
    Commented Jul 17, 2021 at 20:16
  • $\begingroup$ @kjetilbhalvorsen Thanks for the comment. Could you briefly elaborate why? $\endgroup$
    – Ananda
    Commented Jul 17, 2021 at 20:17
  • 2
    $\begingroup$ That is explained elsewhere on the site: stats.stackexchange.com/questions/52976/… $\endgroup$
    – Arya McCarthy
    Commented Jul 17, 2021 at 20:27
  • 1
    $\begingroup$ Covariances are variances. In particular, the eigenvalues are variances (of the eigenvectors). Because variances are expectations of squares and squares (by definition) are never negative, the eigenvalues cannot be negative. $\endgroup$
    – whuber ♦
    Commented Jul 17, 2021 at 20:30
  • 1
    $\begingroup$ Those matrices are explicitly sums of squares, so they too must be positive semidefinite. $\endgroup$
    – whuber ♦
    Commented Jul 17, 2021 at 21:39

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.