Is normalizing the features always good for classification? I always read in books that when we do classification or machine learning tasks it's always better to normalize the features so to make them in one range like 0-1. Today I used weka to play with Iris dataset. First I just built a J48 classifier without normalizing the values, and the it made perfect performance. However when I normalized all the features to be in the range 0-1, the classifier made so much mistakes. Why is that? Shouldn't normalization be used always?
 A: It depends on the algorithm. For some algorithms normalization has no effect. Generally, algorithms that work with distances tend to work better on normalized data but this doesn't mean the performance will always be higher after normalization.
Note that many algorithms have tuning parameters which you may need to change after normalization. What you are seeing may just be that the default parameter settings for J48 happened to work well for the unnormalized data.
A: If your attributes already have a meaningful and comparable scale then normalization can destroy important information.
Take e.g. data coming from a physical experiment. Coordinates are measure in x,y,z, each axis is in milimeters. Since the experiment is performed on a flat dish, x and y vary on the range of 0-100 (i.e. 10 centimeters), but the z axis only varies from 0-10 (i.e. a 1 cm high box).
Normalizing such data with greatly emphasize the z axis, which most likely is not supported by a physical interpretation of the results.
Key point of the story: understanding your data is essential.
Normalization is a hotfix if you don't understand the scales of your data.
A: Typically decision trees (for instance C4.5, implemented as J48 in Weka you used) are non parametric, that is they don't make any assumption regarding the distribution of the data. As long as the normalization doesn't change the ranks of the data (and I know of no normalization that does that), the results will be exactly the same (you will only get different splitting levels).
Of course this doesn't hold for algorithms making parametric assumptions (logistic regression, etc.) So you shouldn't always normalize, but you should decide to do it or not depending on your algorithm.
