4
$\begingroup$

PCA reduces data set dimensions while trying to keep most variations in data set.

PCA can be used as a dimension reducing technique in discrimination, however it tries to keep the most discrimination power rather than keep most variations as more variation does not necessarily mean the separation of different group of data is maximum. So Chang (1983) in his paper "On using principal components before separating a mixture of two multivariate normal distributions" suggest using Mahalanobis distance in some way to measure it. His paper can be found here: http://www.jstor.org/discover/10.2307/2347949?uid=3738032&uid=2129&uid=2&uid=70&uid=4&sid=21100901419681

Does the method work well when the data size are very different (consider two groups only), Say sample sizes have ratio 1:99? If it does not work well, how to handle the problem?

I have read literature about rebalancing data set for PCA, but there are no theoretical and mathematical explanation for how rebalance data can help increase the performance of PCA. Is there any literature for the mathematical explanation part?

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ Let me ask you to clarify and detail your question. First, what is "imbalance"? Groups of very different n or else? So, do you have your sample split into groups or not? Second, PCA (?)tries to keep the most discrimination power rather than keep most variations seems to contradict with PCA trying to keep most variations in data set. Please, elucidate. $\endgroup$
    – ttnphns
    Commented Jul 7, 2012 at 9:28
  • $\begingroup$ Thanks. I have edited it. The question should be more clearly now. $\endgroup$
    – zca0
    Commented Jul 7, 2012 at 10:00
  • $\begingroup$ Is the essence of your question this: There's a multivariate data presumably heterogeneous due to presence of clusters (groups) in it; the clusters are not known, so discriminant analysis is not applicable. Is it then possible to use PCA in place of it, to differentiate the groups? If yes, how to make PCA maximally effective for this discriminative task? Etc. What I recommend you is to rework this and the other (just next) your questions and merge them into one question. $\endgroup$
    – ttnphns
    Commented Jul 7, 2012 at 10:43

1 Answer 1

Reset to default
0
$\begingroup$

I addressed your confusion about PCA in your other question. The correct version of your statement is that PCA attempts to characterize most of the variation in the in a lower dimensional space. As I explain in the answer to the other post PCA is not used for discrimination. It has been used to do clustering after reducing dimension to a subset of the principal components.

If by imbalance of the data you mean one cluster center has many more points arund it than another then that may affect the ability of any clustering method at identifying the smaller cluster but the magnitude of separation of the clusters is probably more important.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.