2
$\begingroup$

Let's suppose I have a given dataset with $n$ features. Having a data-centric approach, I would like to measure the loss of performance of applying a given dimensionnality reduction technique, for a given clustering algorithm.

The dimensionality reduction part doesn't matter to the point that my problem could be seen as a symmetric problem, that is how well do I over/under-perform if I get a dataset with more/less features.

In other words I would like to benchmark several couples of (dimensionality reducer, clustering algo) considered as black-boxes. The only thing I suppose I know, is the dataset I have and its number of features before and after the dimension modification step : $n$ and say $n+m$ where $m\in \mathbb Z$.

Is there any measure of clustering performance that allows me to compare the results of the clustering algorithm before and after the alteration of dimension?

I am aware of a number of clustering performance metrics, I would like to know the 'best' ones in my particular case.

Thanks in advance.

EDIT

This is why "best" is between pythonic quotation marks gung.

As far as i know, clustering rely on a distance (often euclidean). Many metrics also rely on distances, and it is somewhat natural to use the same distance for clustering and evaluating clustering. Some other metrics don't rely on a distance, at the cost of not being able to compare clustering on different datasets (rank index).

The point is to find a metric that meets the following requirements :

  1. different datasets dimensions (with one set included in the other)
  2. different resulting numbers of clusters
  3. eventually different number of points in the datatasets (this in not mandatory)

It should measure "how much" consistent are the clusters given by the augmentation or reduction of the dataset dimension compared to the clustering on the initial dataset.

Silhouette coefficient looks like a good candidate but maybe there is a more appropriate one or just a more sophisticated version.

Let me know

$\endgroup$
  • 2
    $\begingroup$ "Best" is often subjective & a hard question to answer. Can you provide any criteria for deciding one metric is better than another? $\endgroup$ – gung Apr 27 '15 at 11:24
  • $\begingroup$ In line with gung's comment, I think this will have to be a judgment call, or at least, it will require getting much more specific about the real-world problem you're trying to solve. Clustering is an unsupervised-learning method, so you can't evaluate how accurate a selection of clusters is, although if you intend to use the clusters as features for supervised learning, you can measure the resulting accuracy there. $\endgroup$ – Kodiologist Apr 27 '15 at 21:20
1
$\begingroup$

Have a look at:

Estivill-Castro, Vladimir (20 June 2002). "Why so many clustering algorithms — A Position Paper". ACM SIGKDD Explorations Newsletter 4 (1): 65-75.

who noted that clustering is in the eye of the beholder.

There is no such thing as an objectively best clustering, and there is no measure of quality. You can measure how well an output agrees with some other result, or you can check if it optimizes some statistical metric like SSQ. But often, there exist some (maybe slow) algorithm that optimizes this metric - so you end up just checking how similar the algorithms are to your metrics...

If all you want to measure is the impact of dimensionality reduction: run the same algorithm with and without, and compare the results. (You just can't be sure which one is really better...)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.