20
$\begingroup$

I've always thought of logistic regression as simply a special case of binomial regression where the link function is the logistic function (instead of, say, a probit function).

From reading the answers on another question I had, though, it sounds like I might be confused, and there is a difference between logistic regression and binomial regression with a logistic link.

What's the difference?

$\endgroup$
14
$\begingroup$

Logistic regression is a binomial regression with the "logistic" link function:

$$g(p)=\log\left(\frac{p}{1-p}\right)=X\beta$$

Although I also think logistic regression is usually applied to binomial proportions rather than binomial counts.

$\endgroup$
  • 1
    $\begingroup$ What do you mean by logistic regression being usually applied to proportions rather than counts? Suppose I'm trying to predict whether people will attend a party or not, and that for a particular party, I know that 9 people attended and 1 did not -- do you mean that logistic regression takes this as one training example (i.e., this party had a success rate of 0.9), while binomial regression with a link would take this as 10 training examples (9 successes, 1 failure)? $\endgroup$ – raegtin Jul 18 '11 at 1:35
  • $\begingroup$ @raehtin - in both cases it would be $1$ sample/training case, with $(n_i,f_i)=(10,0.9)$ and $(n_i,x_i)=(10,9)$ respectively. The difference is the form of the mean and variance functions. For binomial, mean is $\mu_i=n_ip_i$, the canoncial link is now $\log\left(\frac{\mu_i}{n_i-\mu_i}\right)$ (also called the "natural parameter"), and the variance function is $V(\mu_i)=\frac{\mu_i(n_i-\mu_i)}{n_i}$ with dispersion parameter $\phi_i=1$. For logistic we have mean $\mu_i=p_i$, the above link, variance function of $V(\mu_i)=\mu_i(1-\mu_i)$ and dispersion equal to $\phi_i=\frac{1}{n_i}$. $\endgroup$ – probabilityislogic Jul 18 '11 at 15:18
  • $\begingroup$ With logistic, the $n_i$ is separated out from the mean and variance functions, so can be more easily taken into account via weighting $\endgroup$ – probabilityislogic Jul 18 '11 at 15:20
  • $\begingroup$ Ah, got it, I think I see. Does this mean that they produce equivalent results (simply arrived at from a different way)? $\endgroup$ – raegtin Jul 19 '11 at 19:39
  • 1
    $\begingroup$ @raegtin - I think so. The GLM weights, $w_{i}^{2}=\frac{1}{\phi_i V(\mu_i)[g'(\mu_i)]^{2}}$, are equal in both cases, and the link function produces the same logit value. So as long as the X variables are also the same, then it should give the same results. $\endgroup$ – probabilityislogic Jul 20 '11 at 12:38
4
$\begingroup$

Binomial regression is any type of GLM using a binomial mean-variance relationship where the variance is given by $\mbox{var}(Y) = \hat{Y}(1-\hat{Y})$. In logistic regression the $\hat{Y} = \mbox{logit}^{-1}(\mathbf{X}\hat{\beta})=1/(1-\exp{(\mathbf{X}\hat{\beta})})$ with the logit function said to be a "link" function. However a general class of binomial regression models can be defined with any type of link function, even functions outputting a range outside of $[0,1]$. For instance, probit regression takes a link of the inverse normal CDF, relative risk regression takes as a link the log function, and additive risk models take the identity link model.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.