In most of the literature, it is emphasized that the L2 norm (MSE) gives a higher error when dealing with outliers compared to the L1 norm (MAE).

But what happens when we normalize the data between 0 and 1?

Doesn't normalizing the data reduce L2's sensitivity to outliers since it will have to calculate the square of values which are below 1 (thus resulting in even smaller errors compared to L1)?


Scaling does not change relations between the values, because when scaling you divide all the values by the same constant. Outliers will stay outliers. If there is a big difference between two values $x_1,x_2$ squared and smaller difference between $x_1,x_3$ squared, then after normalizing the values, the differences between them will change, but their relations will be the same. Same with differences of absolute values. Obviously, after normalizing in squared case the distances will still be on squared scale, and in absolute values case, they still will be linear.

| cite | improve this answer | |
  • $\begingroup$ Could you please elaborate more on the term relation between values? or maybe give an example ? thanks... $\endgroup$ – Cypher Jun 30 '17 at 13:23
  • $\begingroup$ @Cypher if you divide all the values by the same constant it does not change the relative distances between them. If you convert meters to kilometers, then still "big" distances will remain big and "small" distances, small. $\endgroup$ – Tim Jun 30 '17 at 14:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.