0
$\begingroup$

I have unlabeled dataset and I'm using the hierarchical clustering to generate a groups from this data. I had a look to the lit and I found that there are two approaches to evaluate the clustering results. (1). Internal Validation, (2) External Validation.

How can I verify that my data clustered well? Is using one of the Internal Validation such as "Sum of Squared Error" enough to say my clustering system perform well? or do I need to apply different criterias?

| cite | improve this question | |
$\endgroup$
  • 1
    $\begingroup$ Internal validation is about "how tight are the clusters inside and/or how much separated are they". External validation is about "to what extent these clusters represent other, predefined classes (classes to which labels the clustering was blind)". The two approaches are independent and additive, not complementary, one cannot compensate for the other. Sometimes one approach of the two is not needed. It depends on your goals. $\endgroup$ – ttnphns Sep 30 '15 at 15:44
  • $\begingroup$ @ttnphns I have no prior knowledge about the groups of the data (unlabeled dataset)! How do you suggest to evaluate the clustering results in this case? I have no ground truth. $\endgroup$ – Omar14 Sep 30 '15 at 15:48
  • 1
    $\begingroup$ Of course you don't have that prior knowledge - it's clustering. I mean - you don't use such knowledge, but you might have it. Suppose I've clustered people by their satisfaction level for a number of products. I observed the dendrogram, I also applied one or several internal clustering criterions; as a result I've chosen 3-cluster and 6-cluster solutions.... $\endgroup$ – ttnphns Sep 30 '15 at 16:16
  • 1
    $\begingroup$ (cont.) I inspected how the clusters in both of them differ in satisfactions for different prodicts and found that both solutions are interpretable. I then go further and find out that clusters in solution "6" substantially differ by external (wrt clustering process) characteristics, such as sex and region. I decide to prefer this solution because it adds facets to the interpretation. $\endgroup$ – ttnphns Sep 30 '15 at 16:20
1
$\begingroup$

Do not rely on internal validation

It does not at all measure how well you e.g. preprocessed your data prior to clustering.

All of these measures are highly sensitive to data preprocessing, and will also usually not compare across different algorithms. Only results of the same algorithm (with mostly the same parameters, although sometimes you can vary e.g. k for k-means) can sometimes be compared to make parameterization easier.

But IMHO, they are largely useless except for synthetic data.

The only thing which will reliably tell you if you have good clusters is to use them, and study them in detail. Often, one cluster is good, but another is not. So don't assume a clustering result gives you a complete image. Usually, you are lucky if you get one idea of what is in your data.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ I find it difficult to concur with your peculiar scepticism towards internal clustering criterions. If they were futile people would have stopped inventing them (100++ versions now). They are weak devices only for those who believe in some universal, paradigm-free clustering algorithm or gold-standard validation. In my reply related to your answer elsewhere I disagreed with your universalistic tendencies (as I perceive it); I said that "biases" of clustering criterions are as inevitable as biases in clustering algos and that it is not nasty. $\endgroup$ – ttnphns Sep 30 '15 at 18:00
  • $\begingroup$ If they would work reliably, people would stop inventing new ones. ;-) Maybe there are 100++ versions because none works on more than one data set? ;-) $\endgroup$ – Has QUIT--Anony-Mousse Sep 30 '15 at 18:16
  • $\begingroup$ If they worked only one one dataset each they would count 100++++ ;-). $\endgroup$ – ttnphns Sep 30 '15 at 18:28
  • $\begingroup$ Not everybody who evaluates a data set is able to come up with a new variation. $\endgroup$ – Has QUIT--Anony-Mousse Sep 30 '15 at 19:49

Not the answer you're looking for? Browse other questions tagged or ask your own question.