I sentiment analisys task, for this I used SVM with an rbf kernel and a linear one. The results for the linear kernel were better than the rbf, from this I know that text is linearly separable, but how can I provide a formal proof of this?.
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
You can't formally prove this, unless you happen to be able to fit a hard margin SVM on your data (unlikely). However, intuitively, text representations are high dimensional (bag of words, n-grams, ...). The higher the dimensionality, the easier it is to linearly separate data, as the VC dimension of a linear classifier in $d$ dimensions is $d+1$ (e.g. see these slides). The VC dimension is the largest amount of points that a classifier can shatter (separate).
Additionally, you should be aware that the linear kernel is equivalent to a degenerate RBF kernel, which means that with a properly tuned RBF kernel you should be getting at least the same accuracy as a linear kernel. That said, using an RBF kernel on such data is a waste of time and effort, because it typically offers very little improvement and tremendously increases computational cost.
answered Jun 5, 2015 at 6:00
-
$\begingroup$ Sure, I tried with the Linear kernel and my metrics increasead amazingly.... Do you think is wrong to use the RBF kernel for text, in particular opinions?. $\endgroup$ Commented Jun 5, 2015 at 6:22
-
1$\begingroup$ @ml_guy it is not wrong per se, as the RBF kernel can yield at least comparable performance. However, training a model with RBF kernel takes a lot more time and you would have to tune the kernel bandwidth, which takes orders of magnitude more time than training a linear model. Hence, using an RBF kernel on high dimensional data is usually a waste of computational effort. You can see a similar reasoning in A practical guide for SVM classification (appendix C). $\endgroup$ Commented Jun 5, 2015 at 6:24