0
$\begingroup$

After reading about both algorithms (Principal Component analysis and Linear Discriminant analysis), I started using them combined in a way which appeared intuitive to me.

I have a data set that I project in 3D using PCA, then I cluster the projected data (e.g. using k-means clustering) and take the biggest cluster as my valid data set and the rest is considered as outliers. Then I use LDA to project my original valid data (not the PCA-projected one) into a space where the separation between classes is maximized. This model is then used to classify any new Input data later. I might also need to keep the PCA model to filter the new input data as well but this is another issue.

My question is: Is it correct to use these algorithms in this way? Or would you suggest a different approach?

$\endgroup$
0
$\begingroup$

Assuming you take only the principal components of your original data, the separation by LDA on that set is practically fine.

It appears to me that you aim at outlier detection. For that purpose, a one-class support vector machine may be suitable on your dataset.

$\endgroup$
  • $\begingroup$ The separation by LDA is done on the original high dimensional data set. I just use PCA as a tool to project my data samples in the space maximizing the variance, thus leading to the fact that very similar samples will form a cluster and outliers will be projected far away from this cluster (my intuition thinking). And then check which high dimensional samples land in the cluster when projected and use them for further processing with LDA. $\endgroup$ – Mehdi Nov 10 '15 at 18:32
  • $\begingroup$ From my understanding, this boils down to a mathematical problem. If you apply it the way described, there should be some sort of transformation of the two methods into each other. I'm interested in the maths behind it, but unfortunately not that firm to do a statement on that idea. What I can do is direct you to PCA as a feature selection step for subsequent classification by LDA (or SVM etc.). Here is a discussion $\endgroup$ – dthettich Nov 11 '15 at 7:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.