5
$\begingroup$

I've found that performing scaling in SVM problems really improves the performance of SVM ... But I don't understand why!

I have read this explanation: "The main advantage of scaling is to avoid attributes in greater numeric ranges dominating those in smaller numeric ranges."

Unfortunately this didn't help me... Can somebody provide me a better explanation?

Thank you in advance!

$\endgroup$
0

2 Answers 2

3
$\begingroup$

To understand the phrase:

"The main advantage of scaling is to avoid attributes in greater numeric ranges dominating those in smaller numeric ranges."

it helps to consider a simple example:

For simplicity, suppose that you are working in a 2-D space, so each observation has two attributes, say $x_1$ and $x_2$. Now let's also suppose that the first attribute $x_1$ takes values such as 1000, 1200, 1100, ... while the attribute $x_2$ takes values that are 100 times smaller, such as 10, 12, 11, ...

We can now see how (and why) the larger values dominate the usual Euclidean measurement of distance (square root of sum of squares), namely $||x|| := \sqrt{x_1^2 + x_2^2}$, as below:

Consider the length of the first point in the example: ||(1000,10)|| = 1,000.050. Notice that the second attribute contributes 0.05 to the length, or less than 0.01% of the total value.

In this example the first attribute dominates so heavily that even a four-fold change in the second attribute will barely register in the distance metric, e.g. ||(1000,40)|| = 1,000.80.

The main point? In order to have a classifier treat attributes as being of equal importance one must (in most cases) normalise the intrinsic difference in the scales with which attributes are measured.

This applies to all classifiers where a distance-type metric is being used, so $k$-means, SVM, and many others as well.


Fine Print: The main point is true in most (but not all) cases; this question is a perfect example of an exceptional case.

$\endgroup$
1
  • $\begingroup$ Thanks for the heads-up. Agree, it's a useful alternative lens on recommender systems. $\endgroup$ Feb 13, 2017 at 7:02
0
$\begingroup$

The statistical techniques based in distances work best with the scaled variables. The basic idea of the SVM is the margin, and this is related with the distance and the scale of the different variables

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.