0
$\begingroup$

This question already has an answer here:

How do we determine when to use a Logistic Regression or a SVM considering the dependent variable is categorical? What are the conditions we need to look at before deciding the either of the two?

$\endgroup$

marked as duplicate by kjetil b halvorsen, mdewey, Peter Flom Feb 19 '18 at 14:00

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

2
$\begingroup$

I recommend you checking:

$\endgroup$
1
$\begingroup$

The main difference is that logistic regression can only separate linearly separable classes where as SVM (with the kernel trick) can find any arbitrarily shaped decision boundary. This means that SVM will usually do better separating your classes (at least on your training set) but is more prone to over-fitting.

Linear regression is also a simpler model with fewer hyper-parameters to tune (zero if you're not using regularization) making it easier to implement.

Unless you have very good intuitions about the separability of your data, I would suggest start by fitting a logistic regression and if it isn't giving you satisfactory class separability, then try an SVM.

On the other hand, if you find that SVM is over-fitting no matter how you tune the hyper-parameters, consider trying logistic regression.

One final point - logistic regression outputs a probability of being in the positive class (you still need to choose a threshold to make it a classifer), SVM just outputs the classes. SVM can give you probabilies via Platt scaling but this can be very slow.

$\endgroup$
  • $\begingroup$ Thanks @Dan!! This is really helpful. Just curious to know a little more on "very good intuitions about the separability of your data". Could you please provide some inputs around this if possible? Is this based on the accuracy score or some other way? $\endgroup$ – Code_Sipra Feb 19 '18 at 11:09
  • $\begingroup$ @Code_Sipra - No I mean if you know your data so well (based on experience / domain knowledge) that you just know what types of models work best. Since I think that is quite rare, instead I suggest going with the simpler, easier model (LR) first and if that turns out to be sufficient then great, if not then try a more complex model (SVM). $\endgroup$ – Dan Feb 19 '18 at 16:21

Not the answer you're looking for? Browse other questions tagged or ask your own question.