3
$\begingroup$

This post Testing for Linear Separability with Linear Programming in R, discusses using linear programming to test if data is linear separable.

What's the connection (if there are any) between LP formulation and logistic regression?

This picture is coming from the link I posted

enter image description here

Specifically, are the lines coming form linear programming as the same as the decision boundary in logistic regression? And why?

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ Your link seems to answer the question completely: because "our objective is to just find a plane and not the the [sic] best plane," the LP approach potentially can return any separating hyperplane. Which one it actually reports would depend on the LP algorithm. When logistic regression also finds a separating hyperplane, it therefore might or might not be the same as the one found by LP. (Indeed, the separating hyperplane found by logistic regression is likely to depend on the details of its algorithm, with different software finding different solutions.) $\endgroup$
    – whuber ♦
    Jan 18, 2017 at 21:48
  • $\begingroup$ See here: stats.stackexchange.com/questions/254124/… for discussion and examples using R package safeBinaryRegression For the theory see ora.ox.ac.uk/objects/uuid:8f9ee0d0-d78e-4101-9ab4-f9cbceed2a2a for kjell konis Phd thesis giving the complete answer to your question! $\endgroup$
    – kjetil b halvorsen ♦
    Mar 26, 2017 at 16:56

1 Answer 1

Reset to default
2
$\begingroup$

No. Deciding if data is separable is a feasibility problem of the SVM without slackness. And it only finds a feasible plane but not an optimal one.

And logistic regression is an unconstrained optimisation problem. If there is no regularization, its solution will not converge ($|\beta|\rightarrow\infty$) (Source:http://pages.cs.wisc.edu/~jerryzhu/cs769/lr.pdf)

Improve this answer
$\endgroup$
1
  • 3
    $\begingroup$ When there is separation in the data but the logistic regression solution does not find it, then the logistic regression solution is wrong. $\endgroup$
    – whuber ♦
    Jan 18, 2017 at 21:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.