Let's say we are doing logistic regression for classification. When the features are used directly, it means we are using some characteristics (features) of the object to classify them. But when using RBF (say with Gaussian kernel), we are using the similarity of an object with some prototype objects as features. But what is the intuition behind this? How does it help?

  • $\begingroup$ Have you heard of the xor problem? What if the mapping between features and class is not linear.? $\endgroup$ – seanv507 Dec 9 '16 at 19:32
  • $\begingroup$ @seanv507, yes. And I (vaguely) understand the reason behind using polynomial basis functions when the relationship is non-linear. I understand mathematically that Gaussian is generalization to polynomial and more powerful since it can go to infinite dimension. However, I fail to understand the intuition. $\endgroup$ – Rakib Dec 9 '16 at 19:36
  • $\begingroup$ so the rbf kernel $k(x,x')=\exp(0.5 \gamma \|x-x'\|²)$ is building a decision boundary by a superposition of circles. place circles around (0,1) and (1,0) and you can solve the xor problem $\endgroup$ – seanv507 Dec 9 '16 at 23:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.