1
$\begingroup$

I have some data that I want to classify. As an initial measure, I did PCA for the data and I saw two distinct clusters of my data. However, when standardizing the data, the two clusters disappear. What can this mean? that the data is easily separated by individual variance or mean of the features? if that is the case, how can I do classification?

Thanks.

Edit: Following the commenters' requests, I am adding an image of my clustered data. Since the PCA is of higher dimension than 3, it is hard to see why classification succeeds from this image. Also, the colors are the TRUE results, not estimated ones.

$\endgroup$
  • 1
    $\begingroup$ Both PCA results are clustering (K-means, in particularly) results are sensitive to standardization of variables. Whether you do PCA/clustering in conjunction or independently, you should first think it over - to do or not to do standardization. And then go ahead with your dicision. $\endgroup$ – ttnphns Nov 9 '14 at 0:19
  • 1
    $\begingroup$ Can you share visualization of the effects you have been observing? Solving your problem with just two sentences of description is hard, it would really benefit from visualization. $\endgroup$ – Has QUIT--Anony-Mousse Nov 9 '14 at 15:32
  • $\begingroup$ Also, "classification" is not "clustering". The first is supervised, the second is un-supervised. Please clarify what you are actually doing. $\endgroup$ – amoeba Nov 9 '14 at 21:55
  • $\begingroup$ @Anony-Mousse I see two distinct clusters (in 3D), and with standartization - I see a single 'messy' cluster with clustering performance of about 50% (i.e. no clustering). $\endgroup$ – yoki Nov 9 '14 at 22:31
  • 1
    $\begingroup$ If the picture above is with log axes, the outliers probably determine your PCA. Is the log meaningful for your data (e.g. is it multiples of something)? In that case you may need to go one step back and think about your data representation first. $\endgroup$ – cbeleites unhappy with SX Nov 11 '14 at 9:51
1
$\begingroup$

Yes there are situations where standardization (mean centering and variance scaling of all variates) can do harm.

See e.g.:

$\endgroup$
  • $\begingroup$ But it shouldn't harm for PCA IMHO. $\endgroup$ – Has QUIT--Anony-Mousse Nov 10 '14 at 18:24
  • $\begingroup$ @Anony-Mousse: I guess that depends on what you consider harming a PCA - a full PCA ultimately reconstructs the complete data set, so all information is still there. But it may cause the interesting information to appear only in higher PCs after meaningless variance that is caused by scaling up originally low (but correlated) signals. $\endgroup$ – cbeleites unhappy with SX Nov 26 '14 at 15:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.