What is the major difference between SVM and Logistic Regression? As both are used for classification purpose and while SVM provides better classification than logistic regression then how is logistic regression advantage over SVM?

  • 1
    $\begingroup$ SVM and logistic regression are not directly comparable, it would make more sense to compare SVM to Perceptrons. Either way, this has been answered a lot - see here and here. $\endgroup$ – Digio Feb 14 '18 at 8:18
  • $\begingroup$ I know that the difference between Logistic regression and SVM is logistic regression find the classifier while SVM finds the classifier with largest margin. It seems that SVM is the best compared to Logistic regression as it provides better classifier. Then why are we still using logistic regression? $\endgroup$ – user195278 Feb 14 '18 at 8:20
  • $\begingroup$ See also stats.stackexchange.com/questions/127042/… $\endgroup$ – kjetil b halvorsen Aug 23 '19 at 23:16

While it's true that SVM may come with higher accuracy, LR is much more than just a "classifier" (if we may call it such at all since it predicts a proportion rather than a class). In short, LR is a parametric/probabilistic method, which produces an inferential and highly interpretable statistical model and, on top of interpretability, it may be used in prediction under certain conditions.

On the other hand, SVM is nonparametric and non-interpretable, and it would be useless in a scenario where you'd care to explain the behaviour and interactions of variables rather than just finding patterns for prediction.

That said, while there are many alternatives to the predictive accuracy of SVM, I can't think of many to the inferential power of LR.

  • 1
    $\begingroup$ "SVM is nonparametric and non-interpretable" is not entirely true... Indeed, SVMs can almost naturally be interpreted as a probabilistic model in the context of Gaussian processes: see mlss2011.comp.nus.edu.sg/uploads/Site/lect1gp.pdf on page 17. $\endgroup$ – Fabian Werner Feb 15 '18 at 12:06
  • 1
    $\begingroup$ Also, why aren't SVM and LR directly comparable? If we select a linear kernel then both, SVM and LR are linear separators... they just optimize w.r.t. a different metric and in that case the SVM is as interpretable as LR... $\endgroup$ – Fabian Werner Feb 15 '18 at 12:07
  • $\begingroup$ From my POV they're not directly comparable because SVM is nonparametric and serves a different purpose, as explained. (Linear) SVM and (Linear) Perceptrons makes much more sense to compare. $\endgroup$ – Digio Feb 15 '18 at 12:25
  • $\begingroup$ The idea behind SVM is completely alien to that of a Gaussian Process. The fact that a SVM can be under certain conditions transformed into a Gaussian Process, is a whole different story. A Deep MLP can be also parametrised and "naturally interpreted" as a simple linear model, but that doesn't mean that neural nets and linear regression are the same thing. For the record, I should also point out that a Gaussian Process, though probabilistic, it is nonparametric and non-interpretable... $\endgroup$ – Digio Feb 15 '18 at 12:34
  • $\begingroup$ Just to make sure I understand the setting, you mean that when you take a MLP without activation function (i.e. every node is just a linear function of the input) then the complete MLP is still a linear function but it is different from a linear regression because the weights were computed w.r.t. a different goal, is that what you want to say? $\endgroup$ – Fabian Werner Feb 15 '18 at 13:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.