Why is feature scaling or weighting important in supervised learning? I can understand that feature scaling or weighting is important in unsupervised learning, because we want a good representation of "similarity". But why is it also important in the supervised learning case like in svm? Supervised learning are supposed to learn those weights. 
 A: I am quite new at this, but I have been following this SVM guide.  It says

Part 2 of Sarle’s Neural Networks FAQ Sarle (1997) explains the importance of this and most of considerations also apply to SVM. The main advantage of scaling is to avoid attributes in greater numeric ranges dominating those in smaller numeric ranges.

The article it refers to quite detailed, here are some excerpts.  Im not sure how many apply just to neural networks tho.

Now the question is, should you do any of these things to your data? The
  answer is, it depends.  
... In particular, scaling the
  inputs to [-1,1] will work better than [0,1]
... It is also bad to have the data confined to a very narraw range such as
  [-0.1,0.1], since most of the initial hyperplanes will miss such a small region. 

The SVM guide also makes the point for numerical difficulties which I believe means overflow.  E.g. if you are using 32-bit floats the intermediate values might exceed ~10^6 if the initial feature values are large.
Also see this video linked from this answer, which explains for the case of gradient decent why features on different scales make the algorithm slower.
Finally, a point that I just thought up -  If all inputs are scaled to a standard range then the training parameters will be consistent across all features. For example in the case of SVMs, gamma is a measure of the influence of each training point.  If the features are scaled, then gamma will refer to the same 'relative distance' for each feature.
A: Robert's answer makes some important points, but here's another aspect: The statement that "feature scaling or weighting is important in surpervised learning" is not generally true.
LDA estimates the within-class covariance and implicitly transforms data such that the covariance is $I$. Pre-scaling features will lead to accordingly scaled LDA weights, but the classification performance will be the same. LDA is invariant, not just under feature scaling, but under arbitrary invertible linear transforms. So in a way LDA does learn "optimal scales" (and "optimal coordinates") by itself.
The same does not hold for SVM. The standard hard-margin SVM maximizes the width of the margin, and this width is measured as the Euclidean distance between the margin borders. The Euclidean distance is not invariant under non-orthogonal linear transforms including feature scaling, and therefore the maximal margin may be found to be reached for different classifier weights (which are not just scaled versions of SVM weights without pre-scaling). The SVM therfore does not learn "optimal scales" by itself.
This means that the necessity to "avoid attributes in greater numeric ranges dominating those in smaller numeric ranges" therefore only holds for some supervised learning methods.
