2
$\begingroup$

I am trying to understand how linear discriminant analysis (LDA) is related to principal component analysis (PCA) and k-means clustering method. As an example, here is a comparison between PCA and k-means:

enter image description here

My question is how LDA is related to PCA and k-means?

$\endgroup$
1
$\begingroup$

I'm by no means an expert in the topic, but it seems that K-means clustering can be viewed as a dimensionality reduction technique, of which LDA and PCA are direct examples. Clustering via K-means seems to uncover the latent structure of data, which essentially results in dimensionality reduction. I'm sure that other people will provide some more advanced answers to this question.

Additionally, I would like to share two references that are relevant to the question/topic and IMHO are rather comprehensive. One reference is a highly-cited research paper by Ding and He (2004) on the relationship between K-means and PCA techniques. Another reference is a research paper by Martinez and Kak (2001), presenting the comparison between PCA and LDA techniques.

References

Ding, C., & He, X. (2004, July). K-means clustering via principal component analysis. In Proceedings of the twenty-first International Conference on Machine Learning (p. 29). ACM.

Martínez, A. M., & Kak, A. C. (2001). PCA versus LDA. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(2), 228-233.

$\endgroup$
  • 1
    $\begingroup$ See stackoverflow.com/a/29731291/2056067 for an attempt to summarize Ding & He. :-) $\endgroup$ – A. Donda May 30 '15 at 16:25
  • $\begingroup$ @A.Donda: Thank you for the link. :-) Both answers are very nice (+1) and I will re-read them, when I'll have a bit more time. However, I think that particular question belongs to either Cross Validated, or Data Science SE site and, therefore, should be migrated. $\endgroup$ – Aleksandr Blekh May 30 '15 at 18:29
  • 1
    $\begingroup$ Maybe so, the line is not very clear for programming-related statistics questions (or vice versa). You can flag the question for moderator attention. DS SE is still in beta though. :) $\endgroup$ – A. Donda May 30 '15 at 23:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.