In the formulation of SVM I don't see any step which states that scaling of features should be done for better generalization performance, nor does it come up in its VC dimension expression. So why is it that SVM's performance gets affected by the feature scaling ?
$\begingroup$
$\endgroup$
2
-
5$\begingroup$ Possible duplicate of Why does the scaling of feature vectors improve performance of SVM classifier? and stats.stackexchange.com/questions/65094/… and stats.stackexchange.com/questions/154224/… $\endgroup$– Sycorax ♦Commented May 16, 2017 at 14:44
-
$\begingroup$ and stats.stackexchange.com/questions/172795/… $\endgroup$– Sycorax ♦Commented May 16, 2017 at 16:03
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
SVM constructs a hyperplane such that it has the largest distance to the nearest data points (called support vectors). If the dimensions have different ranges, the dimension with much bigger range of values influences the distance more than other dimensions. So its necessary to scale the features such that all the features have similar influence when calculating the distance to construct a hyperplane.
Another advantage is to avoid the numerical difficulties during the calculation. Because kernel values usually depend on inner products of features vectors, large attribute values might cause numerical problems.
Referfence: http://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf
