# How do I avoid multiple t-tests (or is it ok to use multiple t tests here?)

I've searched around CV for a bit, and have not found a good answer as to what test to use instead of multiple t-tests in my particular situation.

I am interested in comparing mean size (of fish) between two habitat types for 20 different species. For each species, I want to know if fish are larger on one habitat type vs the other. I do not care about the relationship between species (I know that some species are bigger than others), and so ANOVA has not seemed appropriate. It seems that I could do 20 t-tests to see if some species are larger in one habitat over the other- or perhaps there is a simpler (or more statistically sound) approach?

If I were to go with t-tests, would this be a case of multiple comparisons, even though the species are independent of one another?

There are two approaches to handling data of this nature: fixed effects and mixed effects. The T-test is basically a linear regression model with the size of the fish as an outcome variable and the lake from which the fish was drawn as the indicator of interest (a 0/1 variable). We have to assume the fish you sampled are representative of the population of fish in either pond, even after accounting for species!

The species of the fish is a substantial source of heterogeneity in this case, since some species are always very large (but may be larger in a healthier pond), and some are always very small (but still may be larger in a healthier pond).

By conditioning on the species of fish using fixed or mixed effects, you are ensuring that you are comparing "apples to apples" and can detect small intraspecies differences that are otherwise washed out by the overall larger variability due to species.

A fixed effect, that is a 0/1 indicator for each species of fish in the sample excluding some "referent" group, is a good approach if you have many (perhaps 10 or more) of each species in the sample. If there are a large number of species relative to the overall sample (for instance, 120 fish sampled among 50 different species), a random intercept model for species can handle this variability.

• Thank you, I will look more into random effects modeling to see if this approach will be the best – Kodiakflds Jun 23 '16 at 0:07
• @Kodiakflds, what was your conclusion or solution in the end? From your question I have a feeling you were not interested in inter-species variability, but were just wondering whether the same species is different between two habitats (and you wanted to know this for 20 different species). Right? – Tilen Mar 13 '17 at 22:16
• @Tilen I dabbled a bit with mixed models, but since I wanted to say something about the individual species, I went with multiple t-tests and a Holm's correction for multiple comparisons. – Kodiakflds Mar 15 '17 at 21:58
• @Tilen I would also add that I've seen another strategy published in several ecology papers: If many tests are being conducted, and the number of significant differences are much greater than 5% (Say 15 of the 20 species were bigger in nutrient rich lakes), then authors have ignored the issue of multiple testing, feeling confident that there really is a treatment effect (even if 1 of the 20 was significant by chance alone). – Kodiakflds Mar 16 '17 at 18:20

I agree with @AdamO, you probably must take into account the phylogentic relationship between species, since a given habitat may have larger fish just because larger species inhabit there. In this case you could run a phylogenetic t-test using phytools::phyl.pairedttest in which you could also include the intraspecific variation in body size as standard error.