4
$\begingroup$

I'm performing logistic regression for binary classification in SAS and it outputs all of the coefficients to the logit model.

I was wondering how you would take those coefficients and use them to convert the model into a linear hyperplane in the original space (i.e., hyperplane that splits the space into two classes, positive and negative).

Improve this question
$\endgroup$
1

1 Answer 1

Reset to default
4
$\begingroup$

Logistic regression does not predict the class of an observation, just the probability that it will belong to a given class. So it only defines a set of "parallel" hyperplanes $b_0 + b_1x_1 + \cdots + b_kx_k = A$, where $A$ is any constant. Each choice of $A$ will give a different sensitivity/specificity for the resulting classification. There is an entire set of techniques (usually called "ROC analysis") for trying to get a good value for this cutoff.

Edit

As @cardinal points out in the comment below, $A=0$ is a special case of using ${\rm logit} (p) = 0$, that is $p=0.5$ as a cutoff between the two classes. So it corresponds to the intuitive rule of classifying into the class with the higher predicted probability.

Improve this answer
$\endgroup$
2
  • 2
    $\begingroup$ (+1): A couple comments: (i) The case $A = 0$ is an interesting special case and has an intuitively satisfying interpretation that might be worth remarking on, and (ii) Note that classifying by choosing some $A \neq 0$ is equivalent to ignoring the fitted intercept estimate and instead modifying it to optimize a different criterion conditional on the estimates of the remaining parameters. $\endgroup$
    – cardinal
    Commented Oct 3, 2011 at 16:32
  • 1
    $\begingroup$ @cardinal. i wanted to comment but my reputation doesnt allow me :-(. Can you please elaborate on your 2nd point ?. By default, when we learn logistic regression no bias (intercept) is added. What is the effect of bias term. Is there any paper discussing this ? $\endgroup$
    – Shew
    Commented Apr 19, 2014 at 23:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.