6
$\begingroup$

The books gives some examples about content based recommendation. An example of what I understood is at below.

A movie's attributes are values between $1$ and $10$. The duration attribute gets values between $1$ and $100$. If we use these raw values to calculate distance, duration will dominate purely because of wider range, so we should to normalize that value.

Standardization formulas usually cause values smaller than $1$. It may between $1$ and $-1$. But if I scale values between $1$ and $10$ so how this normalization can be right? I expect that $1$ duration value should be represent $1$ and $100$ duration value should be represent $10$. But as you have know standardization formula cause smaller than $1$. Why it is so?

Do I have to re-scale result for $1-10$ range again? For example if result is $0.43$ so that should represent $4$?

$\endgroup$
11
$\begingroup$

Rescaling the input features is just a linear transformation. There's no right or wrong way of rescaling outside a problem context. If you want to map the range 1 - 100 to the range 1 - 10 linearly you should do: $$ x \leftarrow \frac{x - 1}{99} \times 9 + 1 $$ This maps 1 to 1 and 100 to 10 and it will make the durations have the same range as the other features.

One problem with the method above is if all the durations are clustered between say 40, with just very few outliers close to 100 then most of the range won't be used. Calculating the z-score of each individual feature may be preferable: $$ x \leftarrow \frac{x - \text{mean(x)}}{\text{stddev}(x)} $$ as the transformed features will all have mean 0 and standard deviation 1 and should be more comparable.

$\endgroup$
2
  • $\begingroup$ Good answer. But can you please tell me how i should to choose range for attributes. Should be range 1-10, 0-1 or 1-100. How can i decide about that. $\endgroup$ – Freshblood May 20 '13 at 16:03
  • $\begingroup$ That really depends on the underlying model, although it won't make a difference for linear models in particular (bar numerical precision), e.g. if you do linear regression and use the range 1-100, the coefficients will be 10 times larger than if you used the range 1-10. $\endgroup$ – aristotle May 21 '13 at 10:37
5
$\begingroup$

One way of standardizing variables is to turn each value into a z-score, by taking

$\frac{x - \bar{x}}{sd}$

Doing this, you would only have to do it once; however, this will not result in a range of -1 to 1, the result can be any number. But most values will be between -1 and 1.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.