0
$\begingroup$

I used DHARMa for my residual diagnostics. For two models, the dispersion test is significant even though the rest of the diagnostic output looks good. I am wondering if both my models are correct despite the dispersion test being significant.

I conducted an experiment at three different locations, each containing 4 plots. Each plot was subjected to a different treatment for one consecutive week. During this week, measurements were conducted two times a day (included as time_since_start (h) of experiment) and on each plot a camera trap was placed to observe animal behaviour. The experiment was repeated three times.

I would like to test if the time since the start of the experiment influences the duration of a certain behavioural trait recorded by the camera traps. To test this I created the following model:

glmmTMB(cbind(time_behaviour, total_video_duration- time_behaviour) ~
        Treatment*I(sqrt(Time_since_start)) + Repetition + Location,
        ziformula = ~ 1,family = "betabinomial")

Originally, I had thought to add the “Repetition” and “Location” as random effects in the model. However as n = 3 for both, I had to add them as fixed effects. When checking my model output using DHARMa the plots look good, but the Dispersion test is significant (P = 0.048).

enter image description here

Even though this is only just significant, a similar problem occurs with another model. For example:

glmmTMB(cbind(Nr_shoots_browsed, total_shoots_available- Nr_shoots_browsed) ~
        Treatment*I(sqrt(Time_since_start)) + Repetition + Location,
        ziformula = ~ 1,family = "betabinomial")

enter image description here

Interestingly, the dispersion test of the model above is not significant when I remove all NA’s in the dataset even though these NA’s occur in other variables than the ones used in the model. Removing these rows with NA’s, results in the removal of rows where the total_shoots_available is 1, where all with >1 remain in the dataset. Ecologically, it would be wrong to remove these and therefore I would leave them in. Furthermore, the cbind structure should account for differences in the total_shoot_available.

I have been playing around a lot with these models, but so far I haven’t found a solution yet. When looking at different posts, I am tempted to accept these models but I am not really sure if this is correct.

Now my question is: Is this significant dispersion a problem and if yes what could I do to make this model better?

$\endgroup$
1
$\begingroup$

I'm the developer of DHARMa. This is a pretty common question, and there is a section in the vignette giving guidance on that, but I just realised now that it's not very well placed, so I will change that. Anyway, that's what I wrote in the vignette about the dispersion tests:

A word of warning that applies also to all other tests that follow: significance in hypothesis tests depends on at least 2 ingredients: strength of the signal, and number of data points. Hence, the p-value alone is not a good indicator of the extent to which your residuals deviate from assumptions. Specifically, if you have a lot of data points, residual diagnostics will nearly inevitably become significant, because having a perfectly fitting model is very unlikely. That, however, doesn’t necessarily mean that you need to change your model. The p-values confirm that there is a deviation from your null hypothesis. It is, however, in your discretion to decide whether this deviation is worth worrying about. If you see a dispersion parameter of 1.01, I would not worry, even if the test is significant. A significant value of 5, however, is clearly a reason to move to a model that accounts for overdispersion.

In your specific case:

  • No, it doesn't look as if you have strong overdispersion. You could confirm that by looking additionally at the outputs of testDispersion. Very likely, your problem is mild, and might as well be ignored.

  • Also, note that p <0.05 will occur in 5% of all tests where the model is 100% correct.

  • Let's assume we believe that there is a problem: as the beta binomial has already a variable dispersion, it can't be the dispersion as such that is the problem, but rather you probably have some kind of misfit that shows up in the dispersion test.

My preferred way to proceed would be to plot residuals against all predictors, to check for misfit and possible heteroskedasticity (btw., there also seems to be a bit of heteroskedasticity in the res ~ fitted plot, do you see this?). Heteroskedasticity can be addressed in glmmTMB with the dispformula argument. If you find nothing, I would keep the model as it is.

$\endgroup$
1
  • 1
    $\begingroup$ Your answer has been really helpful and after testing everything again I decided to keep the models like posted. Before posting this question, I carefully read the DHARMa vignette several times. I find its description very clear and therefore I already used the testDispersion and tested the residuals against the predictors (which all look good). I think the sentence "It is, however, in your discretion to decide whether this deviation is worth worrying about." made me doubt myself and therefore wanted to double check. The dispformula was definitely a great tip for further use. Thanks again. $\endgroup$ – SvC Oct 12 '20 at 9:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.