Questions tagged [overdispersion]
Overdispersion is when there is greater variability than there 'ought' to be in the data. Eg, the variance of counts is often greater than the mean, whereas the variance of a Poisson should equal the mean.
0
votes
0answers
8 views
How do I use Tarone's Z-statistic?
Context
In this blog the author suggests using Tarone's Z-statistic to test for overdispersion in a binomial model to determine whether or not it is necessary to use a beta-binomial model instead. In ...
0
votes
0answers
9 views
Overdispersion tests from DHARMa and sjstats: conflicting results?
I ran some models for my count data, and did some diagnostics to check for overdispersion.
Here is a dharma graph, which as I understand, indicates no overdispersion.
And this is the result I get ...
1
vote
0answers
20 views
Adjusting for clustering and overdispersion in count models
My question is specific to the estimation of glm's and correcting for 'clustering' in a quasi-experiment (difference-in-differences). My outcome is counts of crimes....
0
votes
1answer
22 views
DHARMa diagnostics: testDispersion and testZeroInflation interpretation
I have been analyzing count data using Poisson distribution in glmmTMB, and just ran some DHARMA diagnostics. However, there don't seem to be a lot of help online on how to interpret the results. Does ...
5
votes
1answer
53 views
QQ plot doesn't look great - maybe quasipoisson with random effects?
I have several glmer()-models that look like this:
...
0
votes
0answers
10 views
Is there a single discrete distribution that handles over and under dispersion? [duplicate]
I have some count data I am trying to model. The variance is very close to the mean, so the Poisson distribution for the entire data set seems like a good starting point. I have done and it seems to ...
2
votes
2answers
52 views
Poisson model appears overdispersed, but usual recommended approaches don't improve fit
Summary: I am trying to model some count data. I initially attempted to fit a poisson GLM, but diagnostics appear to indicate overdispersion. I have tried several different recommended remedies but ...
0
votes
0answers
10 views
the importance of estimating correctly the dispersion parameter in hypothesis testing
I am reading an article, which is related to negative binomial model in estimating the dispersion parameter.
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0081415
The author ...
0
votes
0answers
35 views
MLE for Overdispersed Poisson
I searched for a while on Google and this website for an answer to this question.
I have an overdispersed Poisson distribution and a "hand-wavy" proof is giving me problems.
Below is the information ...
0
votes
1answer
59 views
Analysing count data for biology
I have twenty observations of three dependent variables and of sixteen independent variables. The dependent variables are: 1) number of species, 2) number of individuals, and 3) index of Shannon. The ...
0
votes
2answers
152 views
Choosing the optimal theta / dispersion parameter for negative binomial regression (glm / glm.nb) in R
I am applying a negative binomial regression to my data in R. For this, I use the package MASS and have two different ways to calculate it:
...
0
votes
0answers
14 views
Overdispersed proportion test?
I am comparing proportions in genomic data. In particular, I compare two populations, A and B, and I look at genetic sites. In other words, I have N contingency tables
...
13
votes
2answers
543 views
Are over-dispersion tests in GLMs actually *useful*?
The phenomenon of 'over-dispersion' in a GLM arises whenever we use a model that restricts the variance of the response variable, and the data exhibits greater variance than the model restriction ...
0
votes
0answers
77 views
Residual Deviance and degrees of freedom - Negative Binomial Distribution
I am trying to model count data using python's statsmodels module (Beer's sold at a football stadium as function of visitors, "tilskuer", and weather data).
...
4
votes
1answer
97 views
How to perform over-dispersion test where null is quasi-Poisson
If I understand correctly, a quasi Poisson regression assumes roughly that
$$
\mbox{E}\left[y\left|x\right.\right] = \exp{\left(x^{\top}\beta\right)},
\quad
\mbox{VAR}\left(y\left|x\right.\right) = \...
1
vote
0answers
95 views
Overdispersion problem in a quasi-binomial GLM (for proportional data)
Below is the summary of a GLM I built for a response variable which is proportional (derived from count data). My only predictor is a continuous one (environmental measurement). And my sample size is ...
1
vote
1answer
33 views
Control for over-dispersion. Why do this: take natural log of metric, exponentiate, rank, remove top and bottom 10%
I'm looking at some NHS healthcare data on the number of deaths in England
The measure i'm looking at is called the SHMI - it's simply:
The number of observed deaths at a hospital / The expected ...
0
votes
0answers
61 views
Relation between mean probability and overdispersion
I have proportion data for N observations x P variables. That is to say for each variable, I have proportions that vary between observations. A proportion is the number of successes divided by the ...
1
vote
0answers
437 views
XGBoost Poisson Objective Function When Data is Over-dispersed [closed]
I am modeling very over-dispersed count data with the goal of prediction. The data is not zero inflated (there are no zeros), but there are a lot of values of 1.
...
1
vote
0answers
59 views
Interpreting Quasi-Linear Regression Predictions
I know that for a simple linear regression the predictions are distributed like:
$$y_i\, |\, x_i\, \sim\, \mathcal{N}\big(\widehat{\beta}_0+\widehat{\beta}_1\, x_i,\ \sigma^2\big)$$
$$\text{where: } \...
0
votes
0answers
11 views
why an alternative model with additional free parameters works?
I am wondering why Gaussian mixture model works? even if you have two free variable, when the sample mean is significantly different from sample variance, the passion dist still fail, right?
[from ...
4
votes
1answer
567 views
Can you use glmmTMB to simultaneously model offsets and zero-inflation?
I'm currently modelling microbial data, with multiple samples and groups of samples. Two problems arise with my data: 1) The data is zero-inflated and dispersed (large variation); 2) Each sample has a ...
0
votes
0answers
182 views
Fitting GLMM to roe deer count data with glmmTMB: workflow and spatial Autocorrelation
I'm fitting GLMM's to roe deer count data derived from GPS locations based on a regular sampling grid with R. From what I could tell glmmTMB seems to be well suited for this kind analysis - especially ...
0
votes
0answers
48 views
How to analyse overdispersed and autocorrelated data from a repeated measures design experiment (Mixed model time-series analysis)?
I have survival (number of alive+number of dead=total number) and productivity (total brood number) data on two ant species made to corxist in lab for 60 days. The question of interest is if group ...
2
votes
0answers
32 views
How to recalculate variance/covariance matrix adding a overdispersion term in glmm?
When models suffer from overdispersion, a solution is to calculate the dispersion parameter (using c_hat or dispersion_glmer in R for example). Then multiply the variance/covarianze matrix of ...
3
votes
1answer
276 views
Adding an observation level random term messes up residuals vs fitted plot. Why?
I run a mixed effects generalized model for proportional data (response variable). I used binomial family and logit link function. I suffered from overdispersion so I added an observation level random ...
0
votes
1answer
591 views
GLM: binomial(logit) with weights=tested
I am trying to decipher what this GLM means for a test:
glm(proportion.correct ~ dose*drug, family=binomial(logit), weights=tested)
The experiment looks at the ...
2
votes
2answers
138 views
binary logistic regression models with correction on over-dispersion [closed]
When using Generalized Lienar Models in SPSS (dist: binomial, link: logit, r/n) how to overdispersion correction? In otherwords how to apply binary logistic regression models with correction on over-...
1
vote
0answers
101 views
Name of the test for over/under dispersion
When testing for over or underdispersion in a count variable there is a test (available for example in the glm.nb() function in the ...
0
votes
1answer
53 views
residual deviance and interactions in quasi-poisson model and negative binomial model?
My data was over-dispersed (dispersion coefficient over 5), so i have fitted both the quasi-poisson model and the negative binomial model. I notice that the regression coefficients are almost the same,...
3
votes
0answers
119 views
Quasi Poisson model quasi-likelihood
Wikipedia (https://en.wikipedia.org/wiki/Variance_function) gives this as the general definition of quasi-(log)-likelihood:
$$ Q(\mu,y)= \int _{y}^{\mu} \frac{y-t}{\sigma^2 V(t)} dt$$
In the case of ...
1
vote
1answer
131 views
Changing from Poisson to NB distribution fixes overdispersion and improves model
I have count data (seconds of behaviour, n=145) with one explanatory categorical variable (Treatment) and 2 interaction terms (Sex and Time). I began by running a GLM with Poisson distribution:
<...
2
votes
0answers
54 views
Non-constant overdispersion (overdispersion parameter depending on covariate) in negative binomial regression
In a negative binomial regression (daily number of cases of a disease over time), after controlling for several factors and experimenting with several functional forms, my results still don't look ...
1
vote
0answers
135 views
Validation of poisson GLMM corrected for overdispersion with OLRE
I am fitting a random effects model to some ecological count data (abundance). The aim is to analyse the effect of influence of chemical pest control on insect population. I run the analysis on ...
6
votes
1answer
489 views
Investigate overdispersion in a plot for a poisson regression
I run a poisson regression by hand and now want to investigate plots whether overdispersion might be a problem or not. What should I plot against each other?
0
votes
0answers
68 views
Overdispersion. Is it ever appropriate to have a non-linear variance correction from Poisson to a Quasi-Poisson?
This is my first Cross Validated post. Please let me know if this is too specific/innapropriate for the site.
I am building a Poisson/quasiposson model with one predictor. The response variable ...
3
votes
1answer
352 views
Overdispersion in Count Data (Poisson model)
There is more than one solution for the problem of overdispersed count data. One is to use a quasipoisson model. One is to use a negative binomial model. One is to use a mixed-level model with subject-...
1
vote
1answer
166 views
how to analyze overdispersed binary data
In my substantive research, I often use dichotomous scoring (1 correct, 0 wrong) for my tests (tests with $15~yes/no$ items). My goal is often to compare the the proportion of correct answers to all ...
1
vote
0answers
106 views
glmm with proportion data and high overdispersion
I have data of time budget behaviors in form of relative frequencies (Nr specific behavior/nr of total behavior observed). The best would be to perform a multinomial analysis, but it seems too hard to ...
1
vote
0answers
70 views
How to intuitively explain the maths behind overdispersion?
I understand that overdispersion indicates extra, unexplained variation in the response than would be expected based on the statistical model of choice. And that the residual deviance is a measure of ...
2
votes
1answer
757 views
Causes for Underdispersion in Poisson Regression
I am working with count data (number of pregnancies per woman), and using glm Poisson (log-link) to model determinants of the former count variable.
From simple descriptives I observe that my data ...
0
votes
2answers
732 views
Quasi-likelihood/Quasi Poisson
I'm facing this new concept: the quasi likelihood. I'm looking for some clear explanation of what it is. I have a very basic knowledge about this, so I need to go step by step very slowly. I ...
1
vote
0answers
432 views
Overdispersion in a binomial GLMER model
I'm having trouble accounting for overdispersion in a binomial GLMER (lme4 package) - I'd read through other posts on the topic but haven't found anything that solves my problem. I tried adding an ...
1
vote
2answers
2k views
The mean and variance of Poisson distribution are equal
This blog claims that the fact that the mean and variance of Poisson distribution are equal can cause problem. Could you please elaborate why this is the limitation and how it can affect models?
2
votes
0answers
56 views
Measure for overdispersion
In statistics, how do you know if your count data is over dispersed after applying a glm, family =poisson to the data. I have 24 results in total, 12 with treatment, 12 without. There are 3 species, ...
2
votes
2answers
167 views
What happen when model selection ranks null model as the best one and there's another model that is competitive?
I'm analyzing the proportion of marked chicks vs. the number of chicks that were recaptured at one moth of age (not possible to use conventional capture-recapture analysis because we don't have a ...
1
vote
1answer
374 views
Overdispersion in Model selection procedures (AIC)
My question is pretty straightforward.
Does overdispersion mean anything when doing model selection and multi model inferences?
I understand that overdispersion affects the estimation of standard ...
3
votes
0answers
410 views
Can including an observation level random effect (OLRE) create nested random factors in a GLMM?
I have data from an observational study on insect parasitism rates from 42 sites. Some sites were sampled only once and others were sampled multiple times across different years. For each sampling ...
1
vote
0answers
86 views
How to find evidence of overdispersion from the deviance table in R?
Using the output , can we find evidence of overdispersion after adjusting for stage? This is a problem from an exercise book.
0
votes
1answer
28 views
How to measure “clumping” in workload
I'm trying to come up with a succinct measurement of the dispersion of employee effort in a services firm.
All employees work on projects. Hypothesis: the fewer projects an employee works on in a ...
