2
$\begingroup$

I was revisiting the differences between logistic regression and Naive Bayes, and had a conceptual question. A logistic regression classifier makes intuitive sense to me as a classifier that directly estimates $P(y \mid x)$ without making any assumptions about how the data is distributed (except that the conditional distribution $P(y \mid x)$ is Bernoulli). To my understanding, however, logistic regression is exactly equal to a Naive Bayes classifier with an assumed Gaussian distribution and constant variance. So, I was wondering, why is it equal if Naive Bayes is making the explicit assumption of a Gaussian distribution?

Thanks.

$\endgroup$
1
  • $\begingroup$ Your assertion would benefit from a numerical illustration, as it is not immediately clear what you mean. The logistic model $p(x)=\frac{1}{1+\exp(-(\beta_0 +\beta_1x))}$ is less Gaussian in nature than the probit model $p(x)= \Phi ( \gamma_0 +\gamma_1x)$ though easier to explain in terms of log-odds $\endgroup$
    – Henry
    Mar 25, 2023 at 10:52

1 Answer 1

1
$\begingroup$

Logistic regression assumes a particular functional form for the conditional probabilities $P(c|\mathbf x)$ ($c$ being the class index).

Under the Naive Bayes assumptions with some further restrictions ($x_i|c$ are independent, gaussian distributed with the same variance), the conditional probabilities $P(c|\mathbf x)$ happen to have the same functional form. So you can view Logistic regression as a generalized model, of which this specific instance of Naive Bayes is a special case. Of course this can also viewed as one of the motivations for choosing this particular functional form.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.