0
$\begingroup$

PCA and LDA are two of the most widely used factor analysis tools. PCA construct the orthogonal basis that capture the maximum variance of the input space, $AP=PD$, where each column of $P$ is an eigenvector.

Since each column is orthogonal by definition, we can project the low-dimension data back to the original dimension. This nice property can serve as a "anomaly detection" tool.

However, LDA does not have this property. The basis we build are not orthogonal anymore. Therefore, I think it does not make sense to project the data back from the low dimension manifold.

I did some experiments: if I back-project the low-dimension 'features' back to the original dimension, the reconstructed data will be hugely different from raw data.

So, can I say that LDA is more a dimension-reduction tool? Surely we can do classification in the low-dimension feature space, but that's not what I want to discuss in this post.

$\endgroup$
  • $\begingroup$ LDA is supervised, and PCA is unsupervised. Maybe you have the methodology mixed up? There are direct analogies between LDA and logistic regression, so maybe that is what you are looking for? $\endgroup$ – Sid Apr 2 '18 at 23:07
  • $\begingroup$ what does orthogonality have to do with "anomaly detection"? $\endgroup$ – Xiaoxiong Lin Oct 29 '18 at 11:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.