Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I'm reading the book "Collective Intelligence" and in one chapter they introduce how to measure similarity between users on a movie review website with euclidean distance.

Now are the movies rated all on the scale from 1-5. But what if I want to find similar users based on features like - lets say body height, body width, weight and ratio of eye-distance to nose-length. This features operate on different scales, so e.g. body height would influence the distance much more than the eye-nose ratio.

My question is what is the best way to approach this example. Should one use a different distance measure (which?), or normalize the data somehow and use euclidean distance?

share|improve this question

3 Answers 3

up vote 3 down vote accepted

A very common solution for this very common problem (ie, over-weighting variables) is to standardize your data.

To do this, you just perform two successive column-wise operations on your data:

  • subtract the mean and

  • divide by the standard deviation

For instance, in NumPy:

>>> # first create a small data matrix comprised of three variables 
>>> # having three different 'scales' (means and variances)

>>> a = 10*NP.random.rand(6)
>>> b = 50*NP.random.rand(6)
>>> c = 2*NP.random.rand(6)
>>> A = NP.column_stack((a, b, c))
>>> A   # the pre-standardized data
    array([[  1.753,  37.809,   1.181],
           [  1.386,   8.333,   0.235],
           [  2.827,  40.5  ,   0.625],
           [  5.516,  47.202,   0.183],
           [  0.599,  27.017,   1.054],
           [  8.918,  35.398,   1.602]])

>>> # mean center the data (columnwise)
>>> A -= NP.mean(A, axis=0)
>>> A
    array([[ -1.747,   5.099,   0.368],
           [ -2.114, -24.377,  -0.578],
           [ -0.673,   7.79 ,  -0.189],
           [  2.016,  14.493,  -0.631],
           [ -2.901,  -5.693,   0.24 ],
           [  5.418,   2.688,   0.789]])

>>> # divide by the standard deviation
>>> A /= NP.std(A, axis=0)
>>> A
    array([[-0.606,  0.409,  0.716],
           [-0.734, -1.957, -1.125],
           [-0.233,  0.626, -0.367],
           [ 0.7  ,  1.164, -1.228],
           [-1.007, -0.457,  0.468],
           [ 1.881,  0.216,  1.536]])
share|improve this answer
    
this seems like an easy to compute normalization - thanks! :) –  tobigue Feb 26 '12 at 14:05
1  
no problem; i should mention that although the terms "standardization", "normalization", and "rescaling" are used more or less interchangeably, what i did above is technically "standardization"; "normalization" (again, the strict definition) means dividing a data row by the vector norm so that its Euclidean length is equal to one. –  doug Feb 26 '12 at 19:44

One option is Gower's generalised (dis)simmilarity coefficient. It is defined as

$$s_{ij} = \frac{\sum \limits_{k = 1}^{m} w_{ijk} s_{ijk}}{\sum \limits_{k = 1}^{m} w_{ijk}}$$

where $s_{ij}$ is the overall similarity between samples $i$ and $j$, $s_{ijk}$ is the similarity between $i$ and $j$ for the $k$th variable and $w_{ijk}$ is the weight applied to the $k$th variable for samples $i$ and $j$.

The $s_{ijk}$ are computed separately for each of the $k$ variables, which is what allows the function to work on data in different units. If the data are binary (0/1) or nomial (classes) then $s_{ijk} = 1$ if and only if both $i$ and $j$ are in the same class or (both present or both absent).

For continuous data, $s_{ijk}$ is computed as

$$s_{ijk} = \frac{1 - |x_{ik} - x_{jk}|}{r_k}$$

where $x$ are the observed data for $i$ or $j$ on the $k$th variable and $r_K$ is the range of the $k$th variable. $r_k$ is an implicit standardisation, again accounting for the fact that each variable is in different units.

The weights $w_{ijk}$ allow additional flexibility and also a facility to handle missing data. If the data are missing for $i$ or $j$ on the $k$th variable then that comparison doesn't contribute to the overall similarity between the two variables as that pairing gets weight 0. If the data are available for $x_i$ and $x_j$ for the $k$th variable then $w_{ijk} = 1$. Weights between 0 and 1 allow the user to give weight to different variables.

share|improve this answer
    
thanks for pointing out this more sophisticated technique. i'll try this and see how it compares to a more easy approach. is range defined as maximum-minimum value among all samples? –  tobigue Feb 26 '12 at 14:08

Despite that there is a huge diversity of proximity measures you could still use euclidean distance in your case. The prerequisite for euclidean distance is interval level of measurement of all variables. All your 4 variables height, width, weight, and the ratio are interval. So, after standardization (such as the one suggested by @doug) of the variables you may apply euclidean distance. In your place, however, I would perhaps consider taking logarithm or angular transformation of the ratio variable first (before standardization). And I don't think you really need Gower coefficient (suggested by @Gavin) in your case, because all your variables are of the same level of measurement.

share|improve this answer
    
thanks for your answer. i'll try the modifications you suggest. :) –  tobigue Feb 26 '12 at 14:09

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.