1
$\begingroup$

In a classification task if we don't care about the exact value of the propabality of the a data point belonging to a class or not, but instead just compare it to a treshold (e.g 0.5) to get a discrete classsification, we just need to check if the input $z = b_{0}+∑(b_{i}X_{i})$ of the sigmoid fucntion $g(z)$ of the Logistic Regression is greater than or equal to zero, that is:

$$ g(z) ≥ 0.5, \text{if z ≥ 0} $$

But, to me this seems to be the exact same thing as training a Linear Regression model and simply checking if the output $Y$ of the model is greater than or equal to zero, if that's the case, why do we need the sigmoid function at all?

$\endgroup$
2
  • 2
    $\begingroup$ This has been discussed at great length on this site on numerous topics. Briefly, use of thresholds in this context is invalid. A meaningful threshold can only be obtained if you already possess a utility/loss/cost function to measure consequences of decisions. Stick with predicted probabilities as continuous measures. $\endgroup$ Commented Dec 17, 2021 at 14:01
  • $\begingroup$ You've discovered that you can make inferences on the logit scale: because $\sigma$ is monotonic increasing and injective, you know that the ordering of values on one scale is the same on the other scale. But this doesn't mean that logistic regression is superfluous: compared to OLS regression, logistic regression produces different parameter estimates for $b_j$, so OLS makes different predictions. See: stats.stackexchange.com/questions/326350/… $\endgroup$
    – Sycorax
    Commented Dec 17, 2021 at 14:05

1 Answer 1

2
$\begingroup$

ALMOST

Remember what a logistic regression is.

$$ \log\bigg( \dfrac{\hat p}{1-\hat p} \bigg) = X\hat\beta $$

When $\hat p =0.5$, $ \log\bigg( \dfrac{\hat p}{1-\hat p} \bigg) = 0 $.

Consequently, if you want to classify based on a threshold of $0.5$, you only need to examine the $X\hat\beta$ portion and check if that value if greater than or less than zero. You can do something similar for any other threshold $\tau$ by calculating $ \log\bigg( \dfrac{\tau}{1-\tau} \bigg) $ and comparing that value to $X\hat\beta$.

The $\hat\beta$ that you calculate for your logistic regression, however, will be different from what you calculate in a linear model, so what you wrote in the question title is not quite correct.

However, the predicted probabilities provide valuable information that you lose when you round them to categorical classifications.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.