6
$\begingroup$

In other words, let's say we have a data representation as in the image below, which is generated from the PCA, the projection of the data onto the first two PCs. As it's shown in the 2-D space, the data is not separable. Is that a good indication that this data is not separable? What other metrics can be used to figure out data separation?

enter image description here

$\endgroup$
6
  • 1
    $\begingroup$ I am not really sure that is a convincing proof of your data not being separable as your plot only covers the first two axes. I already had to deal with datasets where only the use of lower rank axes of PCA could allow for correct discrimination. $\endgroup$
    – Riff
    Jun 2 '16 at 6:53
  • $\begingroup$ Thanks Nicolas, and how can getting a prior indication about that the lower PCs could be the best? Is it just by trial and error strategy? $\endgroup$
    – mhdella
    Jun 2 '16 at 7:13
  • 1
    $\begingroup$ Your graph uses red, green and blue for 3 classes, Difficulty in distinguishing red and green is a common variety of colo[u]r-blindness. colorbrewer2.org suggests colo[u]r schemes that work well more generally. $\endgroup$
    – Nick Cox
    Jun 2 '16 at 8:24
  • $\begingroup$ @mhdella Honestly I would say yes, try and plot some more with 3rd, 4th or even 5th components and try to see if you can better dsicriminate using those. You could even grab all your axis and use variable selection in your discrimination function to get the best combination of components to separate your data. $\endgroup$
    – Riff
    Jun 2 '16 at 11:50
  • 1
    $\begingroup$ PCA isn't for looking at separability. It's for looking at the structure of variance in an entire dataset. For looking at separability, you're probably better off looking at some clustering method. $\endgroup$
    – naught101
    Jun 9 '16 at 23:33
3
$\begingroup$

There is an asymmetry worth noting here.

If a PCA plot shows distinct, separated clusters, then it is clear evidence for the separability of the data. But an absence of this kind of structure in the PCA plot (such as in your example) is not evidence for a lack of separability.

This is because (as pointed out in the comments) your 2-dimensional plot above omits information in the dataset, assuming it contains >2 dimensions. You may simply be looking at the wrong dimensions! There is no rule that says that the data pattern or structure you are interested in must show up in the first two principal components; they are merely the dimensions with the most variation in the dataset. It is entirely possible for dimensions with less variation (i.e. principal components 3, or 4, or whatever) to be the ones on which the data are clearly separable.

If you are interested in separability and identifying the dimensions that contribute to this, PCA might not be the most useful tool. As suggested by @naught101, a clustering approach is likely to be more useful.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.