4
$\begingroup$

I always read in books that when we do classification or machine learning tasks it's always better to normalize the features so to make them in one range like 0-1. Today I used weka to play with Iris dataset. First I just built a J48 classifier without normalizing the values, and the it made perfect performance. However when I normalized all the features to be in the range 0-1, the classifier made so much mistakes. Why is that? Shouldn't normalization be used always?

$\endgroup$
3
  • $\begingroup$ The context is too broad, and you did not state why you would want to optimize an improper accuracy scoring rule such as classification accuracy. You also need to state what problem normalization would attempt to solve. In general normalization gets in the way of understanding more than it helps, but occasionally normalization reduces the dimensionality of the model in a helpful way. $\endgroup$ Oct 31, 2014 at 12:58
  • $\begingroup$ Note that normalization means different things within statistical science. The tag's definition is scaling to $[0,1]$; yours may differ. $\endgroup$
    – Nick Cox
    Oct 31, 2014 at 15:46
  • $\begingroup$ Are you measuring accuracy on the same data you trained on, or on a held-out test set? And how did you normalize down to [0,1]? Both of these can make a difference in your results. $\endgroup$
    – Wayne
    Sep 19, 2016 at 14:48

3 Answers 3

4
$\begingroup$

It depends on the algorithm. For some algorithms normalization has no effect. Generally, algorithms that work with distances tend to work better on normalized data but this doesn't mean the performance will always be higher after normalization.

Note that many algorithms have tuning parameters which you may need to change after normalization. What you are seeing may just be that the default parameter settings for J48 happened to work well for the unnormalized data.

$\endgroup$
4
$\begingroup$

If your attributes already have a meaningful and comparable scale then normalization can destroy important information.

Take e.g. data coming from a physical experiment. Coordinates are measure in x,y,z, each axis is in milimeters. Since the experiment is performed on a flat dish, x and y vary on the range of 0-100 (i.e. 10 centimeters), but the z axis only varies from 0-10 (i.e. a 1 cm high box).

Normalizing such data with greatly emphasize the z axis, which most likely is not supported by a physical interpretation of the results.

Key point of the story: understanding your data is essential.

Normalization is a hotfix if you don't understand the scales of your data.

$\endgroup$
1
  • $\begingroup$ Thanks a lot. I have been thinking about cases where normalization could hurt. Your example makes sense. I could see the performance degrades with normalization, especially if the collected data does not have a uniform distribution across x-axis due to some limitations of the machine. $\endgroup$
    – SaTa
    Jan 20, 2021 at 3:06
1
$\begingroup$

Typically decision trees (for instance C4.5, implemented as J48 in Weka you used) are non parametric, that is they don't make any assumption regarding the distribution of the data. As long as the normalization doesn't change the ranks of the data (and I know of no normalization that does that), the results will be exactly the same (you will only get different splitting levels).

Of course this doesn't hold for algorithms making parametric assumptions (logistic regression, etc.) So you shouldn't always normalize, but you should decide to do it or not depending on your algorithm.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.