2
$\begingroup$

I am really confused about these three techniques. Different articles define each differently.

$$ 1 . \frac{x_i - x_{min}}{x_{max} - x_{min}} $$

$$ 2. \frac{x_i - x_{mean}}{x_{max} - x_{min}} $$ $$ 3. \frac{x_i - \mu}{\sigma} $$

A. Scaling
B. Normalization
C. Standardization

Please advise me which is which.

$\endgroup$
3
  • 7
    $\begingroup$ The question is also the answer. The methods are the important thing. What they're called is not consistent, or even especially important. Never trust that a name means any particular procedure unless you can also see a formula (or even better, the code used). Beyond that, my impression is that statistical people equate C and 3 most readily, while machine learning people are more likely to talk about scaling or normalization. Note also the sense, not included here, that normalization means transforming so that a normal (Gaussian) distribution is a better fit. $\endgroup$
    – Nick Cox
    Dec 2, 2020 at 14:15
  • $\begingroup$ thank you!!!. I got it. $\endgroup$ Dec 2, 2020 at 14:37
  • $\begingroup$ All the three (and many other) methods fall under the label "linear rescaling", because they add (subtract) a constant and then multiply (divide) by a constant. $\endgroup$
    – ttnphns
    Dec 21, 2020 at 18:04

1 Answer 1

5
$\begingroup$

The question is also the answer. The methods are the important thing. What they're called is not consistent across fields or even between experts or experienced people. Given that, the names can confuse as much as they clarify. Facetiously but also seriously, there are many senses regarded by their users as standard or normal that don't match usage elsewhere.

Never trust that a name means any particular procedure unless you can also see a formula (or even better, the code used).

Beyond that, my impressions are that

  1. Statistical people equate C and 3 most readily. Note that standardisation (say (value $-$ mean) / SD) as a linear scaling does nothing to change the shape of a distribution. Thus it leaves the skewness and kurtosis unchanged and so also how far a distribution is from normal or Gaussian (or from any other named reference distribution).

  2. Machine learning people are more likely to talk about scaling or normalization. As I understand it, the main motive for scaling is to get variables on the same footing, which can be somewhere between helpful and essential depending on what is done next. As a silly but illustrative example, if you wanted to group places on their mean annual temperature in $^\circ$C and their mean annual precipitation in mm, the former varies by at most tens but the latter could vary by thousands. Hence results of many clustering algorithms would be dominated by precipitation unless you scaled first. If the example is outside your interests, think of clustering people by their height and weight. Another example found in mainstream statistics as well as machine learning is principal component analysis (PCA) in which a ragbag of variables with different units of measurement should typically be standardized first (or equivalently PCA should be based on the correlation matrix, not the covariance matrix).

  3. There is also the sense, not included here, that normalization means transforming so that a normal (Gaussian) distribution is a better fit.

Otherwise, let us underline that (assuming $x_{max} > x_{min}$)

$$ 1 . \frac{x_i - x_{min}}{x_{max} - x_{min}} $$

yields results in $[0, 1]$ (and the limits will always be present in the results), while

$$ 2. \frac{x_i - x_{mean}}{x_{max} - x_{min}} $$

yields results in $(-1, 1)$ (noting that $-1$ and $1$ are unattainable). In contrast

$$ 3. \frac{x_i - \mu}{\sigma} $$

-- where $\mu$ and $\sigma$ are understood as mean and standard deviation -- produces results with mean 0 and SD 1 and with some values always outside $[-1, 1]$.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.