Timeline for How much dispersion is too much for quasipoisson regression?
Current License: CC BY-SA 4.0
11 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jul 15 at 10:47 | comment | added | rolando2 | @Frans Rodenburg Interesting example, and I appreciate it. I see that in it variance > mean overall while variance ~= mean within groups. The problem becomes which definition of overdispersion to trust. I am still seeing a contradiction between your definition and that of so many others, including the primary answer at stats.stackexchange.com/questions/554622/… | |
| Jul 15 at 6:28 | comment | added | Frans Rodenburg |
Data: set.seed(2024);n <- 1e3;x <- sample(0:1, n, replace = TRUE);y <- rpois(n, exp(-0.5 + 2.5 * x)) Mean and variance within groups are equal: mean(y[x == 0]); var(y[x == 0]); mean(y[x == 1]); var(y[x == 1]) Overall mean differs substantially from overall variance: mean(y); var(y)
|
|
| Jul 15 at 6:28 | comment | added | Frans Rodenburg | @rolando2 Maybe it is easier to understand with a categorical explanatory variable, like categories A and B. Both follow a true Poisson distribution, but the mean of A and B differ. If you look at the counts without knowing which group, they will have a different mean. But since the model can give them each their own mean, they also each have their own variance. So in within groups mean equals variance, while the overall variance is still larger than the mean. In this example, do you still believe there is overdispersion? (Example code in next comment.) | |
| Jul 14 at 15:00 | comment | added | Frans Rodenburg |
@rolando2 We went from 6 to 7 observations... I'm not sure what this is supposed to prove. Try simulating Poisson distributed data with a mean that depends on $x$. Of course you will not have overdispersion, because there is no possible source of overdispersion. And yet, if the mean increases enough over the range of $x$, the variance is going to be larger than the overall mean of $y$. Here is an example: set.seed(1);n=1e3;x=runif(n, 0, 10);y=rpois(n, exp(x));mean(y);var(y);summary(glm(y~x,family="poisson")). Zero overdispersion, variance is $1000 \times$ the mean.
|
|
| Jul 14 at 14:51 | comment | added | rolando2 | Thank you, but it's really not the only exception to your earlier statement, because one can use x=c(10,10,10,10,14,14,15); y=c(3,4,5,5,6,8,10), with mean and variance of y ~5.8, and one still gets the same pattern of results: no overdispersion and a nonzero slope. | |
| Jul 14 at 12:14 | comment | added | rolando2 | Counterexample: We obtain a nonzero slope with no overdispersion when we use x=c(0,0,0,0,4,4), y=c(0,0,1,1,2,2) mean(y); var(y) # 1.0 and 0.8 summary(testq <- glm(y~x,family=quasipoisson())) #Same 0.35 coefficient with poisson regression per se, but using quasipoisson prints out the dispersion parameter of 0.5 | |
| Jul 14 at 12:01 | comment | added | Lukas Lohse | @rolando2 I'm afraid you are misreading the article. They also define things with $\mu$ as a variable, i.e. conditioned on $X$. This is analog to the idea of normal distribution in linear models stats.stackexchange.com/a/12266/341520 | |
| Jul 14 at 11:31 | comment | added | Frans Rodenburg | @rolando2, If the mean-variance relationship of the outcome, unconditional on the explanatory effects mattered, then we would have overdispersion for any Poisson regression with a non-zero slope. | |
| Jul 14 at 11:14 | comment | added | rolando2 | For you to say that the mean/variance ratio is "irrelevant" contradicts many, many sources that explain or define overdispersion, including the Adam O answer you cite and the Ver Hoef/Boveng article I cite. Thus I'm struggling with this statement of yours. | |
| Jul 14 at 7:37 | history | edited | Frans Rodenburg | CC BY-SA 4.0 |
deleted 5 characters in body
|
| Jul 13 at 19:14 | history | answered | Frans Rodenburg | CC BY-SA 4.0 |