enter image description here

I got this plot when I plotted a clustering object in R. If I run km <- clara(data, 2), then plot(km), I get a similar plot. How do we interpret this plot with respect to the clustering? How useful would a similar plot be if I have more than five clusters? To be more precise, how are principal components related to clusters?


Principal components are combinations of features. For example, say you have demographic data consisting of three features: height, weight and income. Then, if height and weight are highly correlated, it might be useful to combine them into a single feature. Principal component analysis (PCA) does this via a weighted linear combination, so you may end up with a feature: 0.5*Height + 0.5* Weight. Now we may find that income is completely independent of height and weight, so Income might be the second principal component discovered.

So say PCA gave us two components: Prin1=0.5*Height+0.5*Weight and Prin2=Income. You can map every point in your dataset to a 2D plot with these two dimension, and it might look something like what you have above.

PCA tries to find combinations of features that lead to maximum separation between data points. What this means is that, if you had another dimension, say age, in your dataset, which was the same for all members, then that would not be considered, alone or in combination, among the top principal components. Only the features that vary a lot from data point to data point form a part of the top principal components. As a result, the points should appear to be quite far apart from each other on the plot.

What happens when you cluster the dataset? Depends on the input feature space and clustering algorithm used. If you use the two components found by your PCA analysis above as input features to your clustering algorithm, a decent clustering algorithm should put points that are close together on your 2D plot into the same cluster. This should happen irrespective of the number of clusters found. If your clustering found more than five clusters, they should still consist of points relatively close together on the 2D plot. So you might get five different circles, somewhat separated from each other.

The plot of PCA vs. clusters may not make sense under a couple of conditions:

a) There is a lot of variance in the dataset along each dimension, so looking at the top 2-3 dimensions does not really give you much information.

b) The clustering algorithm for some reason focuses on features considered unimportant by PCA. Given the variety of clustering algorithms out there, this could happen.

This is a very high-level view of PCA. Take a look at this tutorial for an excellent accessible introduction.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.