28
$\begingroup$

I am trying to model count data in R that is apparently underdispersed (Dispersion Parameter ~ .40). This is probably why a glm with family = poisson or a negative binomial (glm.nb) model are not significant. When I look at the descriptives of my data, I don't have the typical skew of count data and the residuals in my two experimental conditions are homogeneous, too.

So my questions are:

  1. Do I even have to use special regression analyses for my count data, if my count data doesn't really behave like count data? I face non-normality sometimes (usually due to the kurtosis), but I used the percentile bootstrap method for comparing trimmed means (Wilcox, 2012) to account for non-normality. Can methods for count data be substituted by any robust method suggested by Wilcox and realized in the WRS package?

  2. If I have to use regression analyses for count data, how do I account for the under-dispersion? The Poisson and the negative binomial distribution assume a higher dispersion, so that shouldn't be appropriate, right? I was thinking about applying the quasi-Poisson distribution, but that's usually recommended for over-dispersion. I read about beta-binomial models which seem to be able to account for over- as well as underdispersion are availabe in the VGAM package of R. The authors however seem to recommend a tilded Poisson distribution, but I can't find it in the package.

Can anyone recommend a procedure for underdispersed data and maybe provide some example R code for it?

Improve this question
$\endgroup$
9
  • 1
    $\begingroup$ How do you know your data is underdispersed? How are you calculating the dispersion parameter? $\endgroup$
    – Hong Ooi
    Aug 14, 2013 at 10:22
  • 1
    $\begingroup$ It would also help to tell us more about what you are interested in. For linear predictor point estimates and prediction of values, underdispersion rarely is a problem but tests and intervals may be unnecessarily conservative (quasi families would help with that). That said, for a "normal" likelihood approach check out the COM Poisson and other generalized Poisson models. $\endgroup$
    – Momo
    Aug 14, 2013 at 11:46
  • $\begingroup$ @ Hung Ooi:I tested the dispersion with dispersiontest(Poissonmodel, alternative = c("less")) and the test turned out significant. $\endgroup$
    – Sil
    Aug 14, 2013 at 11:47
  • 1
    $\begingroup$ @ Momo: I want to test if negotiating dyads in two experimental conditions differ in the correct offers they make. Correct offers mean that dyads claim more issues that correspond to their teams' respective interests instead of claiming issues more valuabe for the other party. First, I wasn´t even aware that this is count data. Do you mean the Conway-Maxwell-Poisson Distribution by COM Poisson? Thanks a lot already! $\endgroup$
    – Sil
    Aug 14, 2013 at 11:58
  • 3
    $\begingroup$ Thanks for the additional info. Yes, I meant the conway-maxwell poisson. Shmueli & co developed a kindbof generalized linear model for it, there also is an R package if you'd like to try. $\endgroup$
    – Momo
    Aug 14, 2013 at 13:17

4 Answers 4

Reset to default
19
$\begingroup$

The best --- and standard ways to handle underdispersed Poisson data is by using a generalized Poisson, or perhaps a hurdle model. Three parameter count models can also be used for underdispersed data; eg Faddy-Smith, Waring, Famoye, Conway-Maxwell and other generalized count models. The only drawback with these is interpretability. But for general underdispersed data the generalized Poisson should be used. It is like negative binomial for overdispersed data. I discuss this in some detail in two of my books, Modeling Count Data (2014) and Negative Binomial Regression, 2nd edition, (2011) both by Cambridge University Press. In R the VGAM package allows for generalized Poisson (GP) regression. Negative values of the dispersion parameter indicate adjustment for underdispersion. You can use the GP model for overdispersed data as well, but generally the NB model is better. When it comes down to it, its best to determine the cause for underdispersion and then select the most appropriate model to deal with it.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Welcome back! Please register &/or merge your accounts (you can find information on how to do this in the My Account section of our help center), then you will be able to edit & comment on your own question. (Your original account is here.) $\endgroup$
    – gung - Reinstate Monica
    Sep 27, 2016 at 15:15
  • $\begingroup$ Can you perform a generalized Poisson analysis on SPSS? $\endgroup$
    – Grace Carroll
    Sep 5, 2019 at 12:34
  • $\begingroup$ Very helpful, but it seems like VGAM has now disabled underdispersion? $\endgroup$
    – John Madden
    Jan 23 at 19:01
4
$\begingroup$

I encountered an under dispersed Poisson once that had to do with frequency at which people would play a social game. It turned out this was due to the extreme regularity with which people would play on Fridays. Removing Friday data gave me the expected overdispersed Poisson. Perhaps you have the option to similarly edit your data.

Improve this answer
$\endgroup$
2
$\begingroup$

It seems that the solution provided by Joseph Hilbe within the vgam package is no longer available. From the manual of the package: The genpoisson() has been simplified to genpoisson0 by only handling positive parameters, hence only overdispersion relative to the Poisson is accommodated. Some of the reasons for this are described in Scollnik (1998), e.g., the probabilities do not sum to unity when lambda is negative. To simply things, VGAM 1.1-4 and later will only handle positive lambda.

Improve this answer
$\endgroup$
1
$\begingroup$

There are situations where underdispersion coalesces with zero-inflation which is typical for preferred children counts by individuals of both sexes. I haven't found a way to capture this to date

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.