1
$\begingroup$

Is it possible to have the same evaluation performances when comparing some clustering algorithms using many unsupervised evaluation measures instead of a supervised one ?

$\endgroup$
3
$\begingroup$

Evaluating an unsupervised method using supervised measures is somewhat unfair anyway. Obviously, any supervised method can be better.

It actually takes the whole concept of unsupervised methods ad absurdum:

You use unsupervised methods to discover something new, not something you already knew (but didn't tell your program).

So in general, you should actually be particularly interested when a clustering algorithm finds something different than your labels. Because then it means you may have found something new.

Supervised evaluation using class labels is not more than an ad-hoc workaround to somewhat evaluate whether a method could work.

$\endgroup$
2
$\begingroup$

You can't expect anything similar to a supervised error rate without known class labels, however you should take a look at Prediction Strength which determines the quality of a clustering by how stable it is.

My answer to Clustering On Informative Features describes how to compute this value. The idea is to split the data into two halves (train/test sets), generate the clustering rules for both sets and see if each pair of test points in the same test cluster would also be in the same train cluster (when applying the train rule).

$\endgroup$
0
$\begingroup$

I don't see how. In supervised learning you have the correct categories available. So oyu can compute an estimate of error rate with unsupervised learning the correct category is unknown so you can't estimate error rate and can't make the same kind of evaluation that you can do with supervised learning.

$\endgroup$
  • $\begingroup$ So using many unsupervised evaluation measures still don't give an idea about how good the algorithm is performing compared to a supervised evaluation measure. Ok. Then, is it really useful to evaluation a clustering result using an unsupervised measure ? and is using many unsupervised measures more meaningful than using only one unsupervised measure ? $\endgroup$ – shn Sep 18 '12 at 11:35
  • 1
    $\begingroup$ clustering gives you an idea of how the data tend to be grouped. An unsupervised learning measure can determine how many clusters and how well separated they are with a measure such as a ratio of the within cluster variance to the between cluster variance. It can be a way to identify groups even though initially separate groups were not known to exist. Supervised learning should not be compared with unsupervised learnign. They address different problems. $\endgroup$ – Michael R. Chernick Sep 18 '12 at 12:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.