Skip to main content
deleted 15 characters in body
Source Link
kjetil b halvorsen
  • 82.8k
  • 32
  • 201
  • 663

I have a set of data (specifics not important) where the outcome is a count, a set of predictor variables (categorical and continuous), a offset variable and a two level hierarchical structure. I first used Poisson regression to model the outcome, using a random intercept which fits the natural structure of the data.

I have been using the ratio of the sum of pearson chi squared residuals and the degrees of freedom to assess over dispersion (I know this is not a perfect measure for this). Its value was 1.94, but should be equal to 1, so I have explored other methods to account for this.

The majority of suggestions on this site say to use either a generalised poisson model (https://www.ncbi.nlm.nih.gov/pubmed/16389919) or a negative binomial distribution is the most popular suggestion. My understanding for using these is that in both of these suggestions the variance is free to vary from the mean (unlike in the poisson distribution). Therefore in my mind, the models should not be dispersed.

Despite this, my measure for over dispersion is 1.49 and 1.63 in the models respectively (should be equal to 1). My understanding is that these models should account for over dispersion, and so then either my measure is poor, or the models are in fact over dispersed.

To clarify, my questions are:

  1. Can a negative binomial model (or generalised poisson) model be over dispersed?

  2. If it cannot be over dispersed, does this mean my model is fitting poorly in other areas, causing the Pearson Chi Sq/DF > 1?

  3. If they can be over dispersed, then what further measures could be taken to account for over dispersion.

  4. There appears to be more information about this online when NOT using a multi level model. Am I naive in thinking the same methods to account for over dispersion apply when a multi level model is being used.

I can provide more detailed information about my code and data upon request, however I did not feel it necessary, as my query are more about the theory.

Thanks, Alex

I have a set of data (specifics not important) where the outcome is a count, a set of predictor variables (categorical and continuous), a offset variable and a two level hierarchical structure. I first used Poisson regression to model the outcome, using a random intercept which fits the natural structure of the data.

I have been using the ratio of the sum of pearson chi squared residuals and the degrees of freedom to assess over dispersion (I know this is not a perfect measure for this). Its value was 1.94, but should be equal to 1, so I have explored other methods to account for this.

The majority of suggestions on this site say to use either a generalised poisson model (https://www.ncbi.nlm.nih.gov/pubmed/16389919) or a negative binomial distribution is the most popular suggestion. My understanding for using these is that in both of these suggestions the variance is free to vary from the mean (unlike in the poisson distribution). Therefore in my mind, the models should not be dispersed.

Despite this, my measure for over dispersion is 1.49 and 1.63 in the models respectively (should be equal to 1). My understanding is that these models should account for over dispersion, and so then either my measure is poor, or the models are in fact over dispersed.

To clarify, my questions are:

  1. Can a negative binomial model (or generalised poisson) model be over dispersed?

  2. If it cannot be over dispersed, does this mean my model is fitting poorly in other areas, causing the Pearson Chi Sq/DF > 1?

  3. If they can be over dispersed, then what further measures could be taken to account for over dispersion.

  4. There appears to be more information about this online when NOT using a multi level model. Am I naive in thinking the same methods to account for over dispersion apply when a multi level model is being used.

I can provide more detailed information about my code and data upon request, however I did not feel it necessary, as my query are more about the theory.

Thanks, Alex

I have a set of data (specifics not important) where the outcome is a count, a set of predictor variables (categorical and continuous), a offset variable and a two level hierarchical structure. I first used Poisson regression to model the outcome, using a random intercept which fits the natural structure of the data.

I have been using the ratio of the sum of pearson chi squared residuals and the degrees of freedom to assess over dispersion (I know this is not a perfect measure for this). Its value was 1.94, but should be equal to 1, so I have explored other methods to account for this.

The majority of suggestions on this site say to use either a generalised poisson model (https://www.ncbi.nlm.nih.gov/pubmed/16389919) or a negative binomial distribution is the most popular suggestion. My understanding for using these is that in both of these suggestions the variance is free to vary from the mean (unlike in the poisson distribution). Therefore in my mind, the models should not be dispersed.

Despite this, my measure for over dispersion is 1.49 and 1.63 in the models respectively (should be equal to 1). My understanding is that these models should account for over dispersion, and so then either my measure is poor, or the models are in fact over dispersed.

To clarify, my questions are:

  1. Can a negative binomial model (or generalised poisson) model be over dispersed?

  2. If it cannot be over dispersed, does this mean my model is fitting poorly in other areas, causing the Pearson Chi Sq/DF > 1?

  3. If they can be over dispersed, then what further measures could be taken to account for over dispersion.

  4. There appears to be more information about this online when NOT using a multi level model. Am I naive in thinking the same methods to account for over dispersion apply when a multi level model is being used.

I can provide more detailed information about my code and data upon request, however I did not feel it necessary, as my query are more about the theory.

Source Link
AP30
  • 355
  • 3
  • 17

Can a model fitted using negative binomial distribution be over dispersed?

I have a set of data (specifics not important) where the outcome is a count, a set of predictor variables (categorical and continuous), a offset variable and a two level hierarchical structure. I first used Poisson regression to model the outcome, using a random intercept which fits the natural structure of the data.

I have been using the ratio of the sum of pearson chi squared residuals and the degrees of freedom to assess over dispersion (I know this is not a perfect measure for this). Its value was 1.94, but should be equal to 1, so I have explored other methods to account for this.

The majority of suggestions on this site say to use either a generalised poisson model (https://www.ncbi.nlm.nih.gov/pubmed/16389919) or a negative binomial distribution is the most popular suggestion. My understanding for using these is that in both of these suggestions the variance is free to vary from the mean (unlike in the poisson distribution). Therefore in my mind, the models should not be dispersed.

Despite this, my measure for over dispersion is 1.49 and 1.63 in the models respectively (should be equal to 1). My understanding is that these models should account for over dispersion, and so then either my measure is poor, or the models are in fact over dispersed.

To clarify, my questions are:

  1. Can a negative binomial model (or generalised poisson) model be over dispersed?

  2. If it cannot be over dispersed, does this mean my model is fitting poorly in other areas, causing the Pearson Chi Sq/DF > 1?

  3. If they can be over dispersed, then what further measures could be taken to account for over dispersion.

  4. There appears to be more information about this online when NOT using a multi level model. Am I naive in thinking the same methods to account for over dispersion apply when a multi level model is being used.

I can provide more detailed information about my code and data upon request, however I did not feel it necessary, as my query are more about the theory.

Thanks, Alex