# Entropy based on euclidian distances between datapoints / clusters centers?

Is it possible to define any useful entropy or conditional entropy which is based on the distance between datapoint(s) and cluster center(s), instead of basing on the number of points assigned to cluster like it is defined in for example to compute the v-measure ? I am not implying that I want an equivalent to v-measure, but I just wonder if it is possible and maybe useful to define a conditional entropy based on distances.

So, What I'm looking for is a kind of conditional entropy based on distances, which allow to have an idea about the "homogeneity" and "completeness" of clusters, with respect to distances.

-
 Have a look at k-means. It does in fact use a kind of variance of distances from the cluster center. However, you should avoid overfitting. Any such measure is biased, when the algorithm you evaluate already optimized wrt. this measure. – Anony-Mousse Feb 16 '12 at 9:18 I know how kmeans performs. What I was asking for is a kind of conditional entropy based on distances, which allow to have an idea about the "homogeneity" and "completeness" of clusters, with respect to distances. – shn Feb 16 '12 at 10:23 @user995434, I cant answer cause I have never used it, but I can imagine just mapping entropy values to the distances. Entropy is a one-sided bounded measure, as the distances are [0,infty). so just make a 1-1 mapping of a definition that you make. I don't think that it will arise from the basic principles which are different, but it may work as a connected measure that you defined. – Vass Feb 16 '12 at 12:44 @Vass an example of how you would do that, would be appreciated ^^ if you can. – shn Feb 19 '12 at 21:15

You could just say, that the distance between points is $k \log( d_{i,j} + 1 )$ where $d_{i,j}$ is the distance between points so that it resembles entropy measures. But since this is strictly monotonically increasing as a function the clustering will not change, but can affect the inferred number of clusters. It penalizes large distances from creating large effects with the logarithmic increase on the distance.