I'm reading the book "Collective Intelligence" and in one chapter they introduce how to measure similarity between users on a movie review website with euclidean distance.

Now are the movies rated all on the scale from 1-5. But what if I want to find similar users based on features like - lets say body height, body width, weight and ratio of eye-distance to nose-length. This features operate on different scales, so e.g. body height would influence the distance much more than the eye-nose ratio.

My question is what is the best way to approach this example. Should one use a different distance measure (which?), or normalize the data somehow and use euclidean distance?


3 Answers 3


A very common solution for this very common problem (ie, over-weighting variables) is to standardize your data.

To do this, you just perform two successive column-wise operations on your data:

  • subtract the mean and

  • divide by the standard deviation

The rationale of these two operations is to ensure the values have zero-mean (when subtracting the mean in the numerator) and unit-variance (by dividing with the standard deviation in the denominator).

For instance, in NumPy:

>>> # first create a small data matrix comprised of three variables 
>>> # having three different 'scales' (means and variances)

>>> a = 10*NP.random.rand(6)
>>> b = 50*NP.random.rand(6)
>>> c = 2*NP.random.rand(6)
>>> A = NP.column_stack((a, b, c))
>>> A   # the pre-standardized data
    array([[  1.753,  37.809,   1.181],
           [  1.386,   8.333,   0.235],
           [  2.827,  40.5  ,   0.625],
           [  5.516,  47.202,   0.183],
           [  0.599,  27.017,   1.054],
           [  8.918,  35.398,   1.602]])

>>> # mean center the data (columnwise)
>>> A -= NP.mean(A, axis=0)
>>> A
    array([[ -1.747,   5.099,   0.368],
           [ -2.114, -24.377,  -0.578],
           [ -0.673,   7.79 ,  -0.189],
           [  2.016,  14.493,  -0.631],
           [ -2.901,  -5.693,   0.24 ],
           [  5.418,   2.688,   0.789]])

>>> # divide by the standard deviation
>>> A /= NP.std(A, axis=0)
>>> A
    array([[-0.606,  0.409,  0.716],
           [-0.734, -1.957, -1.125],
           [-0.233,  0.626, -0.367],
           [ 0.7  ,  1.164, -1.228],
           [-1.007, -0.457,  0.468],
           [ 1.881,  0.216,  1.536]])

One option is Gower's generalised (dis)simmilarity coefficient. It is defined as

$$s_{ij} = \frac{\sum \limits_{k = 1}^{m} w_{ijk} s_{ijk}}{\sum \limits_{k = 1}^{m} w_{ijk}}$$

where $s_{ij}$ is the overall similarity between samples $i$ and $j$, $s_{ijk}$ is the similarity between $i$ and $j$ for the $k$th variable and $w_{ijk}$ is the weight applied to the $k$th variable for samples $i$ and $j$.

The $s_{ijk}$ are computed separately for each of the $k$ variables, which is what allows the function to work on data in different units. If the data are binary (0/1) or nomial (classes) then $s_{ijk} = 1$ if and only if both $i$ and $j$ are in the same class or (both present or both absent).

For continuous data, $s_{ijk}$ is computed as

$$s_{ijk} = \frac{1 - |x_{ik} - x_{jk}|}{r_k}$$

where $x$ are the observed data for $i$ or $j$ on the $k$th variable and $r_K$ is the range of the $k$th variable. $r_k$ is an implicit standardisation, again accounting for the fact that each variable is in different units.

The weights $w_{ijk}$ allow additional flexibility and also a facility to handle missing data. If the data are missing for $i$ or $j$ on the $k$th variable then that comparison doesn't contribute to the overall similarity between the two variables as that pairing gets weight 0. If the data are available for $x_i$ and $x_j$ for the $k$th variable then $w_{ijk} = 1$. Weights between 0 and 1 allow the user to give weight to different variables.

  • $\begingroup$ thanks for pointing out this more sophisticated technique. i'll try this and see how it compares to a more easy approach. is range defined as maximum-minimum value among all samples? $\endgroup$
    – tobigue
    Feb 26, 2012 at 14:08

Despite that there is a huge diversity of proximity measures you could still use euclidean distance in your case. The prerequisite for euclidean distance is interval level of measurement of all variables. All your 4 variables height, width, weight, and the ratio are interval. So, after standardization (such as the one suggested by @doug) of the variables you may apply euclidean distance. In your place, however, I would perhaps consider taking logarithm or angular transformation of the ratio variable first (before standardization). And I don't think you really need Gower coefficient (suggested by @Gavin) in your case, because all your variables are of the same level of measurement.

  • $\begingroup$ thanks for your answer. i'll try the modifications you suggest. :) $\endgroup$
    – tobigue
    Feb 26, 2012 at 14:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.