I have read that SVM is preferred over logistic regression for skewed data,i know why logistic regression fail for skewed data(have read about F1 score and all), but can't seem to get head around why SVM's work on skewed data.
$\begingroup$
$\endgroup$
6
-
5$\begingroup$ Source for the claim? Neither is preferred, they excel in different things. Now, logistic regression really do penalize over all the support, while SVMs only penalize on/across the margin, so perhaps this might explain that claim. $\endgroup$– FirebugCommented Sep 4, 2017 at 20:09
-
1$\begingroup$ What exactly is "skewed data"? $\endgroup$– Matthew DruryCommented Sep 4, 2017 at 20:13
-
$\begingroup$ suppose in data set of 1000 only 20 are positive and rest are negative ,so even without training logistic regression like for example even if we all time give answer as negative on new data , though we might be wrong but our success rate is high as their are very few positives in our data set,so even if our trained logistic regression gives false positives success rate is gonna be high(but by F1 scores we can test our algo),i read somewhere that SVM would perform better on such data set,but why? $\endgroup$– Divyanshu daiyaCommented Sep 4, 2017 at 20:27
-
3$\begingroup$ Logistic regression predicts probabilities, not class assignments. So all you will find in that case is that the predicted probabilities are all small, which aligns well with the data. There is no reason to believe svm would do better, what you read is mistaken and overgeneralizing. $\endgroup$– Matthew DruryCommented Sep 4, 2017 at 20:29
-
1$\begingroup$ thanks for help man ,Mathew Drury,i thought same way, but wasn't able to assure myself $\endgroup$– Divyanshu daiyaCommented Sep 4, 2017 at 20:33
|
Show 1 more comment
1 Answer
$\begingroup$
$\endgroup$
2
I believe you are talking about class weighted SVM implementations that will give higher penalty to misclassified labels belonging to smaller class. This definitely is preferred over Logistic regression which carries no such notion of assigning weights. The issue you mention is called unbalanced classes, not skewed data. Also, in such cases, I would suggest you to focus on recall metric also.
-
3$\begingroup$ the same thing can be done in logistic regression as well (as LR is often fitted using iteratively re-weighted least-squares giving different weights to different patterns ought to be fairly straighforward) $\endgroup$ Commented Sep 5, 2017 at 9:23
-
$\begingroup$ In the Coursera machine learning class by Andrew Ng/Stanford, this is called "Skewed Classes", so I guess that notion will spread. A bit unfortunate, since I think Skewness is a better term for ordinal data rather than categorical data. $\endgroup$– LudvigHCommented Dec 12, 2018 at 15:58