I'm performing clustering on a labeled dataset. I would like to check the quality of clustering. Is there a well accepted way of doing that?
So basically I would like perform some classification-like procedure, where I would determine the quality of the clustering, since I already have the labeled dataset. Does that make sense and how to do it?
There are many many measures that can be used on labeled data.
For example, if you run k-means wiht $k=3$ on the mouse data set:
you get the following evaluation result (using ELKI):
Clearly, it did not work very well. If you know this toy data set, k-means just doesn't work well on it, because the clusters have too different size.
These are external evaluation measures. They work well if the labels correspond to clusters. If you are using classification data, the labels may not at all correspond to clusters; but some classes may form one big cluster, or a class may split into multiple clusters. There may also be outliers. They work well on synthetic data, but real data just never has such labels already.
So while such measures are a nice thing for experimenting, they have big issues... IMHO, their results can be totally misleading. A clustering algorithm that works perfectly well may score really bad on such a measure, if the labels do not correspond to the data clustering structure.
Clustering just is not classification. It's rather orthogonal.
They work well if the labels correspond to clusters. It may sound as if the correspondence is an assumption under which one may apply an external evaluation measure. But the whole task of the evaluation really is to try to discover that correspondence - the one that maximizes the measure - and then to see how big is that maximized measure.
$\endgroup$
If your data is labeled that is the true classification of your data set. Then you can apply any of the know clustering methods (hierarchical, kmeans or model-based clustering) and use the adjustedRandIndex. This is a function in R in the mclust package. Adjusted Rand Index indicates how similars the clusters are, and when the value is 1 it means that they are identical. Hence you will compare the tue clustering with the result of the different clustering methods. There is another function in R that does approx the same thing it is call Error Rate or something like this and it computes the proportion of the truly identified clusters from the clustering method you have chosen.
Clustering is usually used for unsupervised classification - that is, when you are trying to discover groups that might exist but that you don't know about.
If you want to classify units into known groups, you could look at multinomial logistic regression or at classification trees and related methods.
