Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I'm trying to get a handle on the concept of overdispersion in logistic regression. I've read that overdispersion is when observed variance of a response variable is greater than would be expected from the binomial distribution.

But if a binomial variable can only have two values (1/0), how can it have a mean and variance?

I'm fine with calculating the mean and variance of successes from x number of Bernoulli trials. But I cannot wrap my head around the concept of a mean and variance of a variable that can only have two values.

Can anyone provide an intuitive overview of:

  1. The concept of a mean and variance in a variable that can only have two values
  2. The concept of overdispersion in a variable that can only have two values
share|improve this question
1  
Add 20 values of $y$, where 10 are $0$ and 10 are $1$. Can you divide this by 20? Can you compute the s.d. $y$? –  user777 Mar 27 at 19:53
    
Nicely put so I believe that's mean = 0.5, standard deviation = 0.11. –  luciano Mar 27 at 20:23
    
Say my response variable had 100 successes and 5 fails. Is this likely to be overdispersed? –  luciano Mar 27 at 20:25
    
luciano, you need more than one realization of the experiment to determine if it is overdispersed. –  Underminer Mar 27 at 20:32

2 Answers 2

up vote 5 down vote accepted

A binomial random variable with $N$ trials and probability of success $p$ can take more than two values. The binomial random variable represents the number of successes in those $N$ trials, and can in fact take $N+1$ different values ($0,1,2,3,...,N$). So if the variance of that distribution is greater than too be expected under the binomial assumptions (perhaps there are excess zeros for instance), that is a case of overdispersion.

Overdispersion does not make sense for a Bernoulli random variable ($N = 1$)

In the context of a logistic regression curve, you can consider a "small slice", or grouping, through a narrow range of predictor value to be a realization of a binomial experiment (maybe we have 10 points in the slice with a certain number of successes and failures). Even though we do not truly have multiple trials at each predictor value and we are looking at proportions instead of raw counts, we would still expect the proportion of each of these "slices" to be close to the curve. If these "slices" have a tendency to be far away from the curve, there is too much variability in the distribution. So by grouping the observations, you create realizations of binomial random variables rather than looking at the 0/1 data individually.

share|improve this answer
1  
But I'm interested in overdispersion in the context of logistic regression. For each value of a predictor variable in logistic regression, there isn't n trials, there is only one trial. And the result of that one trial is either success or fail –  luciano Mar 27 at 20:39
    
I just added a paragraph to address the intuition behind overdispersion in the context of linear regression. –  Underminer Mar 27 at 21:25
    
Underminer, I'm trying to imagine what you mean by this sentence: "If these "slices" have a tendency to be far away from the curve, there is too much variability in the distribution". Here's what I think you mean: at the slice on the curve where there is say a 0.1-0.3 probability of success there are lots of successes and at the slice on the curve where there is say a 0.7-0.9 probability of success there are lots of fails. Is this what you mean and would this represent overdispersion? –  luciano Mar 28 at 6:32
    
@luciano That is the right idea. But keep in mind there has to be a balance of "slices" that are too far above and too far below the curve in order for the fit to have occurred in the first place. So it may be more realistic to say that a slice around 0.7 has too many successes (maybe 100%) and the next slice around 0.75 has too few (50%) then 0.80 has too many (100%), etc. So there is more variance observed than would be expected. –  Underminer Mar 28 at 14:52
    
I've got ya, well explained –  luciano Mar 28 at 15:16

As already noted by others, overdispersion doesn't apply in the case of a Bernoulli (0/1) variable, since in that case, the mean necessarily determines the variance. In the context of logistic regression, this means that if your outcome is binary, you can't estimate a dispersion parameter. (N.B. This does not mean that you can ignore potential correlation between observations just because your outcome is binary!)

If, on the other hand, your outcome is a set of proportions, then you can estimate a dispersion parameter (which, although often greater than one, can also be less than one) by dividing the Pearson chi-squared statistic (or the deviance) by the residual degrees of freedom.

Remember, logistic regression with a purely binary outcome is just a special case of the more general logistic regression model in which the binomial index can exceed one (and can vary across observations). Thus, the question of whether you're fitting a logistic regression model or not is unrelated to the question of whether your data are overdispersed.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.