4
$\begingroup$

A binomial GLM can be written as:

$Y_{i}\thicksim B(1,\pi_{i})$

$\mathrm{E}(Y_{i})=\pi_{i}$ and $\mathrm{var}(Y_{i})=\pi_{i}\times(1-\pi_{i})$

$\mathrm{logit}(\pi_{i})=\mathrm{intercept} + \beta \times treatment_{i}$

1) If we consider a simplest model without any covariates but only an intercept. Given a number of sample $n$, the standard error of the expected probability $\pi_{i}$ (of outcome) would be $$\sqrt{\frac{\pi_{i}\times(1-\pi_{i})}{n}}$$.

This indicates that when $\pi_{i}=0.5$, we obtain the highest SE, thus largest confidence interval.

2) On the other hand, if we apply a logistic regression (maybe with a continuous variable $x$) and plot the predicted probability with its confidence interval against $x$. Does it mean that at $\pi_{i}=0.5$ the confidence interval is always the widest, or narrowest? (see an example figure I copied from internet). I have an impression that the prediction curve often shows a narrow band in the middle.

enter image description here

Can someone give me more information about statement 2) and how to link 1) and 2)?

3) It is often said that when the number of events and non-events are equal, the logistic regression performs the best. This statement is often addressed to avoid complete separation problem when using maximum likelihood estimation in the logistic regression. Can it also be linked to statement 1) and 2)?

$\endgroup$
  • $\begingroup$ I don't understand your plot. What is Bid? How is it related to an intercept only model? $\endgroup$ – Erik May 26 '15 at 12:43
  • 2
    $\begingroup$ By the way, my off-hand idea is that you don't account for the fact that logistic regression models log-odds instead of success probabilities. If you properly transform one into the other, binomial estimation and the logistic regression give the same result. $\endgroup$ – Erik May 26 '15 at 13:02
  • $\begingroup$ @Erik. I took a pic from internet and now I find another one, I am not sure if my statement 2) is correct. I come up with this question, because very often a see a prediction curve with a narrow band in the middle and I assume that's around $\pi=0.5$. $\endgroup$ – tiantianchen May 26 '15 at 13:16
1
$\begingroup$

Some pointers that I hope adress your question:

  1. It is true that when estimating the success probability for a simple binomial distribution the standard error is smallest when the probability is 0.5. Since it's really the same thing this includes intercept-only logistic regression, but note that logistic regression usually operates on the log-odds scale so you will need to transform the predictions to success probabilities.
  2. Consider simple linear regression. The confidence bands grow wider as you begin to extrapolate outside the range of data observed. For example, if you built a model using subjects age 30-40 your error bands will be larger for age = 60 than for age = 35. That is because you extrapolate and is caused by correlation of the intercept and slope. See Wikipedia.

When you consider a logistic regression with a predictor then you get a combination of both effects. For example, your plot shows the narrowest band at a bit below 0.5. That is due to the distribution of AGE.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.