When prepocessing continuous and integer data, is it better to normalize to $[0, 1]$ (i.e., $\frac{x-\min(x)}{\max(x)-\min(x)}$) or normalize via z-score (i.e., $\frac{x-\bar{x}}{s_x}$)? Is doing both a bad practice?

Would you suggest other normalization techniques?


It depends on your input.

If you have for example 8-bit RGB channels, then you know that the minimum and maximum of any future data will always be 0 and 255. In such cases, normalizing is clearly the better choice.

If you have data with a central tendency, and a variance, both of which you expect to be somewhat similar in future data, then you should standardize using the meand and standard deviation of the train set.

Doing both makes no sense: The second scaling option undoes the changes introduced by the first.

  • $\begingroup$ Would you agree in normalize to [0,1] when the variables are not normally distributed and standardize when they follow a normal distribution? or is it better to not use both of these methods at the same time and just use one of them. $\endgroup$ – Nip Oct 1 '19 at 3:19
  • 3
    $\begingroup$ Normalization rests on the assumption that you can reasonably estimate the extremes. Depending on your sample size, there is a good chance you have never observed them. In fact, many distributions have at least one extreme that is $\infty$ or $-\infty$. In such cases, you may want to standardize, or use something different entirely. As to using mixed variables, I would scale each variable on its own with whatever seems appropriate. $\endgroup$ – Frans Rodenburg Oct 1 '19 at 3:27
  • $\begingroup$ I've read that normalization via min-max changes the range of the data, and normalization via z-score changes the shape of the data. I think using both could be against the main objective - dont over weigh a particular feature just because its values are much larger than other features. This also has to be with the use of euclidean distance formula. $\endgroup$ – Nip Oct 1 '19 at 19:18

Any normalization can be wrong, and will usually be inferior to an informed choice of scaling parameters. It is only a common choice to get a first impression of data and to get results quickly.

For example, you should not use it on any kind of histogram (it should be normalised to a total sum of 2 instead, not per bucket!). It is also inappropriate to use it on TF-IDF vectors, here normalizing to a L2 norm of 1 is more appropriate. When your data is skewed, also both will usually fail, and also when your data has outliers (then the scaling will be based on the outliers, not the data!). In some cases, you really should first do a log transform or a box-cox transform. Last but not least, don't forget more robust alternatives, for example $$\frac{x-Median(X)}{MAD(X)}$$

  • $\begingroup$ Frans & Anony. Some algoriths may benefits from normalization via min-max and others from normalization via z-score. Would you use these transformation in simplier models like OLS, or LOGIT? or is it better to use robust models? $\endgroup$ – Nip Oct 1 '19 at 17:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.