# How to handle data imbalance in Principal Component Analysis?

PCA reduces data set dimensions while trying to keep most variations in data set.

PCA can be used as a dimension reducing technique in discrimination, however it tries to keep the most discrimination power rather than keep most variations as more variation does not necessarily mean the separation of different group of data is maximum. So Chang (1983) in his paper "On using principal components before separating a mixture of two multivariate normal distributions" suggest using Mahalanobis distance in some way to measure it. His paper can be found here: http://www.jstor.org/discover/10.2307/2347949?uid=3738032&uid=2129&uid=2&uid=70&uid=4&sid=21100901419681

Does the method work well when the data size are very different (consider two groups only), Say sample sizes have ratio 1:99? If it does not work well, how to handle the problem?

I have read literature about rebalancing data set for PCA, but there are no theoretical and mathematical explanation for how rebalance data can help increase the performance of PCA. Is there any literature for the mathematical explanation part?

-
Let me ask you to clarify and detail your question. First, what is "imbalance"? Groups of very different n or else? So, do you have your sample split into groups or not? Second, PCA (?)tries to keep the most discrimination power rather than keep most variations seems to contradict with PCA trying to keep most variations in data set. Please, elucidate. – ttnphns Jul 7 '12 at 9:28
Thanks. I have edited it. The question should be more clearly now. – zca0 Jul 7 '12 at 10:00
Is the essence of your question this: There's a multivariate data presumably heterogeneous due to presence of clusters (groups) in it; the clusters are not known, so discriminant analysis is not applicable. Is it then possible to use PCA in place of it, to differentiate the groups? If yes, how to make PCA maximally effective for this discriminative task? Etc. What I recommend you is to rework this and the other (just next) your questions and merge them into one question. – ttnphns Jul 7 '12 at 10:43