4
$\begingroup$

I sentiment analisys task, for this I used SVM with an rbf kernel and a linear one. The results for the linear kernel were better than the rbf, from this I know that text is linearly separable, but how can I provide a formal proof of this?.

$\endgroup$
2
$\begingroup$

You can't formally prove this, unless you happen to be able to fit a hard margin SVM on your data (unlikely). However, intuitively, text representations are high dimensional (bag of words, n-grams, ...). The higher the dimensionality, the easier it is to linearly separate data, as the VC dimension of a linear classifier in $d$ dimensions is $d+1$ (e.g. see these slides). The VC dimension is the largest amount of points that a classifier can shatter (separate).

Additionally, you should be aware that the linear kernel is equivalent to a degenerate RBF kernel, which means that with a properly tuned RBF kernel you should be getting at least the same accuracy as a linear kernel. That said, using an RBF kernel on such data is a waste of time and effort, because it typically offers very little improvement and tremendously increases computational cost.

$\endgroup$
  • $\begingroup$ Sure, I tried with the Linear kernel and my metrics increasead amazingly.... Do you think is wrong to use the RBF kernel for text, in particular opinions?. $\endgroup$ – tumbleweed Jun 5 '15 at 6:22
  • 1
    $\begingroup$ @ml_guy it is not wrong per se, as the RBF kernel can yield at least comparable performance. However, training a model with RBF kernel takes a lot more time and you would have to tune the kernel bandwidth, which takes orders of magnitude more time than training a linear model. Hence, using an RBF kernel on high dimensional data is usually a waste of computational effort. You can see a similar reasoning in A practical guide for SVM classification (appendix C). $\endgroup$ – Marc Claesen Jun 5 '15 at 6:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.