Can unsupervised evaluation measures for clustering replace a supervised evaluation measure?

Question

Is it possible to have the same evaluation performances when comparing some clustering algorithms using many unsupervised evaluation measures instead of a supervised one ?

Answer 1

Evaluating an unsupervised method using supervised measures is somewhat unfair anyway. Obviously, any supervised method can be better.

It actually takes the whole concept of unsupervised methods ad absurdum:

You use unsupervised methods to discover something new, not something you already knew (but didn't tell your program).

So in general, you should actually be particularly interested when a clustering algorithm finds something different than your labels. Because then it means you may have found something new.

Supervised evaluation using class labels is not more than an ad-hoc workaround to somewhat evaluate whether a method could work.

Answer 2

You can't expect anything similar to a supervised error rate without known class labels, however you should take a look at Prediction Strength which determines the quality of a clustering by how stable it is.

My answer to Clustering On Informative Features describes how to compute this value. The idea is to split the data into two halves (train/test sets), generate the clustering rules for both sets and see if each pair of test points in the same test cluster would also be in the same train cluster (when applying the train rule).

Answer 3

I don't see how. In supervised learning you have the correct categories available. So oyu can compute an estimate of error rate with unsupervised learning the correct category is unknown so you can't estimate error rate and can't make the same kind of evaluation that you can do with supervised learning.