0
$\begingroup$

I have a clustering results by different unsupervised algorithms in the form of coordinates (x,y) and class (it's currently binary). I would like evaluate the quality of clustering by some value. Most of the used techniques assuming that the closely located entries in the plot are related, and if they are from different classes that's not good.

I have checked the R packages like clusterSim and clv (cluster evaluatin) by all of them seems to work with original data, but not with the already produced coordinates.

Can you please suggest me the quantitative evaluation technique for described situation?

PS: My data is imbalanced with ratio around 1:10

$\endgroup$
3
$\begingroup$

all of them seems to work with original data, but not with the already produced coordinates.

I don't understand this sentence. The don't work with the original data, but with the original data?

Anyway, I believe you are trying to do external evaluation, but you are looking at internal evaluation methods.

Internal evaluation uses the coordinates (or usually the distances) of objects to compute e.g. variances and cluster separation. External evaluation on the other hand compares the resulting clustering with a reference solution (i.e. existing class labels)

Both have their benefits and drawbacks.

  • With internal evaluation you do see overfitting. One method (e.g. k-means) may just naturally optimize for a criterion similar to the one you use for evaluation, whereas another algorithm (e.g. DBSCAN) does not optimize for any such statistical criterion at all. Results will be biased to methods such as k-means by design; internal evaluation should not be used with different algorithms, but is best used to compare multiple runs of the same algorithm (e.g. running k-means multiple times).
  • Internal evaluation is sensitive to preprocessing.
  • At the same time, internal evaluation doesn't need class labels, so you can evaluate without having existing classes.

  • External evaluation obviously can only be used when you have labeled data.

  • External evaluation allows for arbitrarily shaped clusters, and is not susceptible to methods overfitting on the evaluation statistic.
  • External evaluation has problems on the motivation side: if you already had labels, why would you use clustering in the first place? Just use the labels!
  • External evaluation actually punishes algorithms that discover interesting sub- or superstructures of the labels. Any deviation from the labels is considered bad, even if it were interesting to a human user...
  • External evaluation cannot handle hierarchical clustering results, as e.g. obtained from hierarchical clustering or OPTICS.

As for external evaluation, I recommend looking at the "pair counting" evaluation methods. Maybe choose the adjusted rand index (ARI) there.

Oh, and make sure to remove labels and shuffle your data before running clustering. I've seen people run clustering with the binary class label as extra attribute... guess what happened...

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.