5
$\begingroup$

I have always been a bit confused about logistic regression modeling. If I speak to a statistician, they refer to the modeling efforts in terms of the logit link function. However, if I speak with a machine learning expert, they refer to logistic regression as simply implementing a hyperplane (similar to SVM). What is the relationship between the logistic function and the separating hyperplane?

$\endgroup$
5
$\begingroup$

The picture below will probably answer your question.

A logistic regression model with 2 features creates a wave based on the logit link function.

enter image description here

Applying the decision rule (for example above 50%) transforms the wave to a separating hyperplane like that, but not similar to, one found in SVM. This is illustrated in the picture below. Note that this separating hyperplane is in feature space.

enter image description here

These pictures come from http://blog.data-miners.com/2014/03/lines-and-circles-and-logistic.html You can find an expose on om the subject.

$\endgroup$
  • 1
    $\begingroup$ This does not answer the question. The question specifically asks about the separating hyperplane. This plane is just the prediction function. $\endgroup$ – Sycorax Dec 10 '15 at 20:23
  • $\begingroup$ @user777 Was adding that, and I think done. $\endgroup$ – spdrnl Dec 10 '15 at 20:27
  • $\begingroup$ Imposing a decision rule on the prediction equation does not achieve the SVM separating hyperplane. The SVM separating hyperplane exists in the feature space of the kernel function; there is not necessarily anything planar about the separation in the space of the original predictors. This is what people mean when they say the SVM is a nonlinear classifier. $\endgroup$ – Sycorax Dec 10 '15 at 20:34
  • 1
    $\begingroup$ @user77 I agree and I know, and I don't think that that is the question. I'm stating the fact that the logit function gives rise to a separating hyperplane, not similar to one in SVM, but related to. This is how the two terms can be related to each other. $\endgroup$ – spdrnl Dec 10 '15 at 20:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.