2
$\begingroup$

Why may log(natural logarithm) transformation improve results of SVM prediction(regression, eps-svm)? Is SVM based on the assumption of normal distribution or something else?

update1. I use Radial basis function kernel.

$\endgroup$
0
4
$\begingroup$

SVM doesn't assume normality. But it's still a regression that minimizes some symmetric loss function (I suppose you use symmetric kernel).

So... this is just a feeling and I'm too tired to justify/prove all this but:

  1. Probably your output variable has highly skewed distribution;
  2. And you use symmetric gaussian kernel that leads to symmetric squared loss to minimize (squared error with bump cut-off if I remember correct?);
  3. Then SVM still estimates something close to conditional mean of your data if you minimize this loss for original output variable;
  4. When you log-transform output variable and minimize that symmetric loss for it, then in terms of original variable it estimates something like a conditional median;
  5. it's well-known that mean is the thing that minimizes average squared error and median is the thing that minimizes average absolute error, so when you estimate regression using log-transformed output you get worse MSE but better MAPE.

Hope this helps.

$\endgroup$
0
$\begingroup$

I presume that you use Support Vector Machines and for each non-zero element of each feature vector you change its weight from n to something like log(1+n). In this case your system will give more importance to the quantity of the different features, that are associated with any class, than to their weight.

For example, you have to classify comments to positive and negative. The word "like" is associated with positive ones, but sometimes it may repeat many times in a negative text ("sounds like trash, like the worst ever seen stuff, like the waste of time"). And if you make such a weighting, your system will give more importance, that there are many negative-associated words, like "trash", "worst" and "waste", that even the highest weight of one single feature "like".

$\endgroup$
2
  • $\begingroup$ I have regression problem, not classification. Before using eps-svm I'm made such transform A = log(A). After prediction - inverse transform for predicted values. As result better MAPE and a bit worse MSE. Your example is still relevant in this case? I can't figure it out. $\endgroup$
    – luckyi
    Oct 2 '13 at 8:19
  • $\begingroup$ Yes, it's still relevant even for regression. ^__^ $\endgroup$
    – Felix
    Oct 2 '13 at 8:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.