3
$\begingroup$

This post Testing for Linear Separability with Linear Programming in R, discusses using linear programming to test if data is linear separable.

What's the connection (if there are any) between LP formulation and logistic regression?

This picture is coming from the link I posted

enter image description here

Specifically, are the lines coming form linear programming as the same as the decision boundary in logistic regression? And why?

$\endgroup$
  • 2
    $\begingroup$ Your link seems to answer the question completely: because "our objective is to just find a plane and not the the [sic] best plane," the LP approach potentially can return any separating hyperplane. Which one it actually reports would depend on the LP algorithm. When logistic regression also finds a separating hyperplane, it therefore might or might not be the same as the one found by LP. (Indeed, the separating hyperplane found by logistic regression is likely to depend on the details of its algorithm, with different software finding different solutions.) $\endgroup$ – whuber Jan 18 '17 at 21:48
  • $\begingroup$ See here: stats.stackexchange.com/questions/254124/… for discussion and examples using R package safeBinaryRegression For the theory see ora.ox.ac.uk/objects/uuid:8f9ee0d0-d78e-4101-9ab4-f9cbceed2a2a for kjell konis Phd thesis giving the complete answer to your question! $\endgroup$ – kjetil b halvorsen Mar 26 '17 at 16:56
2
$\begingroup$

No. Deciding if data is separable is a feasibility problem of the SVM without slackness. And it only finds a feasible plane but not an optimal one.

And logistic regression is an unconstrained optimisation problem. If there is no regularization, its solution will not converge ($|\beta|\rightarrow\infty$) (Source:http://pages.cs.wisc.edu/~jerryzhu/cs769/lr.pdf)

$\endgroup$
  • 3
    $\begingroup$ When there is separation in the data but the logistic regression solution does not find it, then the logistic regression solution is wrong. $\endgroup$ – whuber Jan 18 '17 at 21:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.