Let's say I run a linear regression model with a binary dependent variable. If I ran logistic regression on the same data would the results be comparable or exactly similar? By results I mean both the beta values and the value of dependent variable. If not why? Also what can I say about linear regression being a subset of Logistic regression or vice versa?

  • $\begingroup$ The models for linear regression and binary logistic regression are both special cases of the generalized linear model (GLM): Linear regression with an identity link and Gaussian conditional distribution (with equal variance), logistic regression with a logit link and binomial conditional distribution. $\endgroup$ – caracal Nov 28 '12 at 13:05
  • $\begingroup$ I discussed the nature of logistic regression in a slightly different context here: difference between logit and probit models. That answer also briefly mentions the connection b/t linear regression and the generalized linear model. It may be helpful to read it. $\endgroup$ – gung - Reinstate Monica Nov 28 '12 at 15:30

Neither method is a subset of the other. Suppose you have response variable $Y$ using covariates $X$. In linear regression, you assume that $E[Y|X]$ is linear in the parameters $\beta$, whereas in logistic regression you assume the log-odds,

$$ \log\Bigg(\frac{P(Y=1|X)}{P(Y=0|X)}\Bigg) $$

is linear in the parameters $\beta$. For each method you assume two different quantities are linear in $\beta$. The estimates of $\beta$ will likely be different and have different interpretations as well.

It may or may not make sense to use logistic regression for your data, but linear regression is almost certainly not a good way to analyze your (dichotomous) data.

| cite | improve this answer | |
  • $\begingroup$ Agreed...Is there any other way to estimate the parameter other than MLE technique in logistic regression ? $\endgroup$ – Mukul Dec 3 '12 at 11:09
  • $\begingroup$ You can use gradient descent with a specific objective function. Andrew Ng has several online lectures available on this topic, how to set up the procedure, its assumptions, etc. $\endgroup$ – Sycorax Jun 4 '13 at 17:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.