Distance function of clustering vs. distance function of evaluating

Question

I want to experiment with different clustering methods and parameters. Especially, with different distance functions (euclid, cityblock, ...). But how can I compare the use of different distance functions by measures like the silhouette coefficient? I have to apply a distance function again, to calculate this coefficient. It seems non-consistent to me, to use distance function A to cluster and distance function B to evaluate. But on the other hand, I am not sure if it makes any sense to compare coefficients that are based on different distance functions as I noticed that for example the cityblock-silhouette coefficient tends to be lower then the euclid-silhouette coefficient.

I think I should use the same distance function when evaluating the cluster results. Maybe a complete new one that I didn't use for the clustering method itself?

How can I overcome these problems? Have you any ideas and advices?

Kind regards

Answer 1

Evaluation scores based on distance should only be used with whatever distance is most appropriate for your problem and data set.

You cannot compare results across different distances. For example, if you compare euclidean d9 0 istance with squared euclidean distance, the results obviously will not be the same, although the distance functions are highly similar (and even monotone, so the nearest neighbor with one will be the nearest neighbor with the other). But clearly, (b-a)/max(a,b) is different from (b²-a²)/max(a²,b²).

Your best way out is to use an external evaluation method that is not based on distance!

Use internal evaluation measures only for tuning simple hyperparameters, such as the random seed or maybe "k" for k-means. And for some basic quality control: if the silhouette is less than 0.5, the clustering probably is not very good.