3
$\begingroup$

I'm performing logistic regression for binary classification in SAS and it outputs all of the coefficients to the logit model.

I was wondering how you would take those coefficients and use them to convert the model into a linear hyperplane in the original space (i.e., hyperplane that splits the space into two classes, positive and negative).

$\endgroup$
4
$\begingroup$

Logistic regression does not predict the class of an observation, just the probability that it will belong to a given class. So it only defines a set of "parallel" hyperplanes $b_0 + b_1x_1 + \cdots + b_kx_k = A$, where $A$ is any constant. Each choice of $A$ will give a different sensitivity/specificity for the resulting classification. There is an entire set of techniques (usually called "ROC analysis") for trying to get a good value for this cutoff.

Edit

As @cardinal points out in the comment below, $A=0$ is a special case of using ${\rm logit} (p) = 0$, that is $p=0.5$ as a cutoff between the two classes. So it corresponds to the intuitive rule of classifying into the class with the higher predicted probability.

$\endgroup$
  • 2
    $\begingroup$ (+1): A couple comments: (i) The case $A = 0$ is an interesting special case and has an intuitively satisfying interpretation that might be worth remarking on, and (ii) Note that classifying by choosing some $A \neq 0$ is equivalent to ignoring the fitted intercept estimate and instead modifying it to optimize a different criterion conditional on the estimates of the remaining parameters. $\endgroup$ – cardinal Oct 3 '11 at 16:32
  • 1
    $\begingroup$ @cardinal. i wanted to comment but my reputation doesnt allow me :-(. Can you please elaborate on your 2nd point ?. By default, when we learn logistic regression no bias (intercept) is added. What is the effect of bias term. Is there any paper discussing this ? $\endgroup$ – Shew Apr 19 '14 at 23:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.