Skip to main content
added 232 characters in body
Source Link
Marc Claesen
  • 18.7k
  • 2
  • 55
  • 76

You can't formally prove this, unless you happen to be able to fit a hard margin SVM on your data (unlikely). However, intuitively, text representations are high dimensional (bag of words, n-grams, ...).

  The higher the dimensionality, the easier it is to linearly separate data, as the VC dimension of a linear classifier in $d$ dimensions is $d+1$ (e.g. see these slides). The VC dimension is the largest amount of points that a classifier can shatter (separate).

Additionally, you should be aware that the linear kernel is equivalent to a degenerate RBF kernel, which means that with a properly tuned RBF kernel you should be getting at least the same accuracy as a linear kernel. That said, using an RBF kernel on such data is a waste of time and effort, because it typically offers very little improvement and tremendously increases computational cost.

You can't formally prove this. However, intuitively, text representations are high dimensional (bag of words, n-grams, ...).

  The higher the dimensionality, the easier it is to linearly separate data, as the VC dimension of a linear classifier in $d$ dimensions is $d+1$ (e.g. see these slides).

Additionally, you should be aware that the linear kernel is equivalent to a degenerate RBF kernel, which means that with a properly tuned RBF kernel you should be getting at least the same accuracy as a linear kernel. That said, using an RBF kernel on such data is a waste of time and effort, because it typically offers very little improvement and tremendously increases computational cost.

You can't formally prove this, unless you happen to be able to fit a hard margin SVM on your data (unlikely). However, intuitively, text representations are high dimensional (bag of words, n-grams, ...). The higher the dimensionality, the easier it is to linearly separate data, as the VC dimension of a linear classifier in $d$ dimensions is $d+1$ (e.g. see these slides). The VC dimension is the largest amount of points that a classifier can shatter (separate).

Additionally, you should be aware that the linear kernel is equivalent to a degenerate RBF kernel, which means that with a properly tuned RBF kernel you should be getting at least the same accuracy as a linear kernel. That said, using an RBF kernel on such data is a waste of time and effort, because it typically offers very little improvement and tremendously increases computational cost.

Source Link
Marc Claesen
  • 18.7k
  • 2
  • 55
  • 76

You can't formally prove this. However, intuitively, text representations are high dimensional (bag of words, n-grams, ...).

The higher the dimensionality, the easier it is to linearly separate data, as the VC dimension of a linear classifier in $d$ dimensions is $d+1$ (e.g. see these slides).

Additionally, you should be aware that the linear kernel is equivalent to a degenerate RBF kernel, which means that with a properly tuned RBF kernel you should be getting at least the same accuracy as a linear kernel. That said, using an RBF kernel on such data is a waste of time and effort, because it typically offers very little improvement and tremendously increases computational cost.