I have a simple task regarding SVM and logistic regression, even though I know the theory behind SVM and logistic regression I still have a problem to solve them.
Exercise. We want to learn a hard-margin linear SVM from the following points, where green points have positive label and red points have negative labels.Find the linear support vector machine? What will you get by training a logistic-regression classifier on the given dataset?
Solution: In order to solve SVM let's find $w$ and $b$. There are two hyper-planes with the largest margin, the first one goes through points $(1,-1)$ and $(1,1)$, the second hyper-plane goes through $(-1,0)$ and parallel to the first one. I hope it's correct?
We get three equaltions
$(-1, 0) \cdot w + b = -1$
$(1, -1) \cdot w + b = 1$
$(1, 1) \cdot w + b = 1$
This transforms to
$-w_1 + b = -1 $
$w_1 -w_2+ b = 1 $
$w_1 + w_2+ b = 1 $
I cannot find exact solution, however by substitution we can check that $b=0$ and $w^T=(1,0)$ does the work. I am not sure if this is correct.
I am highly confused by the second question: What will you get by training a logistic-regression classifier on the given dataset?
I would start by defining margins according to sigmoid function.
$w_1 + b < 0$; $-2w_1 + 2w_2 +b <0$; $-2w_1 - 2w_2 +b <0$
$w_1 -w_2 + b > 0$;$w_1 +w_2 + b > 0$;$2w_1 -2w_2 + b > 0$;$2w_1 + b > 0$;$2w_1 +2w_2 + b > 0$
I am not sure how to proceed from here.
Here I have two questions, whether the used approach to solve SVM is correct? How to answer the second question regarding logistic regression?