My question: do I need to be concerned with multiple comparisons when fitting models to different (but related) sets of count data?

My question expanded: I have a series of plots where we collected plant abundance data. Thus, we went to specific points on the landscape and counted the number of individuals of a targeted set of species (not all the species). We repeated this at three dates and are interested in whether (and how) densities (individual counts) of each species changed over time and the relationship with environmental conditions at each plot.

Reproduceable example: The Salamander data in the R package glmmTMB works as an example,

# pull out data on two species for this example
ECL <- Salamanders[Salamanders$spp == "EC-L",]
DESL <- Salamanders[Salamanders$spp == "DES-L",]
# Test the effect of mining on abundance of each species, separately, incorporating a 
# random site effect
ecl.m <- glmmTMB(count ~ mined + (1|site), zi=~mined, ECL, family = "poisson")
desl.m <- glmmTMB(count ~ mined + (1|site), zi=~mined, DESL, family = "poisson")
# excerpted output

# Conditional model:
#   Estimate Std. Error z value Pr(>|z|)    
# (Intercept)  -1.8691     0.4962  -3.767 0.000165 ***
#   minedno       3.0109     0.5637   5.342 9.22e-08 ***

# again, just main model

# Conditional model:
#   Estimate Std. Error z value Pr(>|z|)  
# (Intercept)  0.03267    0.83821   0.039    0.969  
# minedno      1.33374    0.77681   1.717    0.086 .

Here I've got two models each estimating the effect of mining on a different species but whose count data were collected in the same sites. We have a significant effect of mining on density of the species "EC-L" and a suggestion (p<0.1) of an effect for "DES-L".

Would these two models be considered multiple comparisons in which I should apply something like a Bonferroni adjustment? This has been suggested to me by a reviewer.

It looks like this question suggests I'm ok as-is: Multiple ANOVAs, then multiple Tukey HSDs, correcting for multiple comparisons But I think this answer is suggesting it would be good to do the correction: Correcting for multiple comparisons after multiple ANOVAs

But, then again, they are different in that I am model-fitting with glmmTMB and, for each species, choosing the model with the lowest AIC to report (thus each final model has a different suite of environmental variables: elevation, exposure, etc...).

Thanks in advance for any advice.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.