# Which machine learning algorithms get affected by feature scaling?

Which of the following machine learning algorithms will be affected if we apply feature scaling?

1. Naïve-Bayes
2. k-Nearest Neighbor (KNN)
3. Support Vector Machine (SVM)
4. Decision Trees
5. Neural Network (NN)
• KNN? You can imagine that scaling a feature will bring all the features onto comparable scales. Failing to do this may make some points seem farther than they actually are. This is my educated guess. Commented Apr 4, 2019 at 0:30
• Good question, could be extended to a library / overview. Commented Apr 4, 2019 at 13:22

KNN algorithm is seriously affected because you choose the $$K$$ closest samples for your predictions. If one of the features has large values (e.g. $$\approx$$ 1000), and the other has small values (e.g. $$\approx 1$$), your predictions will favor the feature with large values because the distance calculated will be dominated with it.
SVM is affected because in the end you're trying to find a max-margin hyperplane separating the classes (or for making regressions). For example, if $$\mathbf{x_1}$$ and $$\mathbf{x_2}$$ are support vectors, we are interested in maximizing the distance between them, i.e. $$||\mathbf{x_1-x_2}||$$. Elements of these vectors are features. And, if we don't want some large features dominating the distance formulation, scaling is necessary.
In Naive Bayes, the critical formula affected by features is the (naive) likelihood $$P(x|C_i)=\prod p(x_j|C_i)$$. The probability distribution of features is not affected by the scaling, since it is one-to-one, we'll have $$p(X_j=x_j|C_i)=p(X_j'=x_j'|C_i)$$ where apostrophe indicates scaled version of the variable. For example, in typical Bag of Words representation, you don't scale the features; on the contrary they've special meanings as the counts of each word.