2
$\begingroup$

In clustering, one has to choose a distance metric. I've seen Normalized Euclidean Distance used for two reasons:

1) Because it scales by the variance.

2) Because it quantifies the distance in terms of number of standard deviations.

However, I have never seen a convincing proof of 2) nor a good explanation of 2).

How does it quantify the standard deviation in each dimension over which you are clustering?

$\endgroup$
  • $\begingroup$ Which normalized Euclidean distance? There are at least two. $\endgroup$ – Anony-Mousse Sep 15 '15 at 13:23
  • $\begingroup$ The one that normalizes based on variance. cytospec.com/pics/NormEuclid.jpg $\endgroup$ – Sother Sep 15 '15 at 16:09
1
$\begingroup$

A general version of this is the Mahalanobis distance. In one dimension you center your variables and then scale them by the standard deviation. The definition of the distance between two points with this metric in one dimension is: $$ d(x,y) = \frac{|x-y|}{s} $$ where $s$ is the standard deviation. So basically you are scaling by the standard deviation and thus one unit in this metric is one standard deviation in the regular Euclidian metric.

This interpretation is possible for vectors if they have coordinates in one dimension or if the standard deviation is the same along all coordinates. Otherwise the unit is not exactly interpretable as standard deviations, but it is normalized so people tend to talk about it as standard deviations in general.

$\endgroup$
0
$\begingroup$

This "normalized Euclidean distance" is a redundant (and expensive) equivalent to simply standardizing your data set. I don't think you need it - just preprocess your data set instead! Also, "standardized Euclidean distance" would be a better name...

More precisely, let for every dimension $i$:

$$u_i := \frac{x_i}{\text{std}(x)}$$

be the new coordinate of object $x$.

Then obviously "normalized Euclidean distance" is equal to regular Euclidean distance on the standardized data.

Lets add a small twist to this. Let's assume we had a symbol $\sigma$, which we will treat as a unit in the physical sense of minutes, seconds, feet, ... we will call this unit "standard deviations". Let's use

$$u_i^\prime := \frac{x_i}{\text{std}(x)}\sigma$$

then the deviation in each attribute is now measured in $\sigma$, or "standard deviations". So this intuitively makes sense as a unit, doesn't it?

If we now compute Euclidean distance on the projected $u^\prime$ vectors, the resulting distance will too have the unit "standard deviations"! So indeed we can argue that we can quantify distance in "standard deviations" now.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.