When setting a treshold to use Logistic Regression for a classification task isn't it the same thing as using Linear Regression?

Question

In a classification task if we don't care about the exact value of the propabality of the a data point belonging to a class or not, but instead just compare it to a treshold (e.g 0.5) to get a discrete classsification, we just need to check if the input $z = b_{0}+∑(b_{i}X_{i})$ of the sigmoid fucntion $g(z)$ of the Logistic Regression is greater than or equal to zero, that is:

$$ g(z) ≥ 0.5, \text{if z ≥ 0} $$

But, to me this seems to be the exact same thing as training a Linear Regression model and simply checking if the output $Y$ of the model is greater than or equal to zero, if that's the case, why do we need the sigmoid function at all?

Answer 1

ALMOST

Remember what a logistic regression is.

$$ \log\bigg( \dfrac{\hat p}{1-\hat p} \bigg) = X\hat\beta $$

When $\hat p =0.5$, $ \log\bigg( \dfrac{\hat p}{1-\hat p} \bigg) = 0 $.

Consequently, if you want to classify based on a threshold of $0.5$, you only need to examine the $X\hat\beta$ portion and check if that value if greater than or less than zero. You can do something similar for any other threshold $\tau$ by calculating $ \log\bigg( \dfrac{\tau}{1-\tau} \bigg) $ and comparing that value to $X\hat\beta$.

The $\hat\beta$ that you calculate for your logistic regression, however, will be different from what you calculate in a linear model, so what you wrote in the question title is not quite correct.

However, the predicted probabilities provide valuable information that you lose when you round them to categorical classifications.