0
$\begingroup$

I know this topic is kind of a recorrent question. I want to evaluate clustering methods, however I don't know which method to use because they are so many. There is precision and recall, there are external validity indices like jaccard, rand , ari, folkes-mallow, etc.. and there is the correct classification rate (CCR).

I think precision, reccal and f1 are more used in the information retrieval context and not clustering.

For evaluation of clustering methods should I use the external indices? Or should I use CCR? Is CCR considered a external validity indice?

(Assuming I know the real clusters in data with simulated data)

$\endgroup$
1
$\begingroup$

Probably the most widely used method is the Adjusted Rand Index (ARI). The CCR (and some others that you list) would assume that you know how to match the found clusters to the true ones, which is not necessarily trivial, particularly not if the numbers of clusters are not identical. I prefer something like the ARI because it doesn't depend on matching.

There is some recent criticism of the ARI (this guy has published a number of papers on that issue, and there is further work) and some alternative methods, although I have to say that in pretty much all instances where I have seen the ARI in use (partly in parallel to other approaches) I have found the ARI to do a good job (as far as I can say), and conclusions didn't seem to depend on whether the ARI or something else was used, so I think it will be fine in most cases.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.