What test should I run if I have measurements nested in mice nested in litters?

Question

I am looking for some clarity on which statistical test I should run. My experiment consisted of administering medication/control to 10 groups of mice to evaluate their neurodevelopment. I have evaluated their neurodevelopment by an open field test, a novel object test, a social interaction test, and an anatomical MRI. 4/10 groups received an injection prenatally and 6/10 received an injection postnatally. Each groups of mice represents a new litter of mice. In the prenatally exposed group the mother received the injection, thus all the babies received the same medication. In the postnatal half the litter received the treatment or the control. I also purchased 2 pregnant mice at the same time to be able to do my experiments similatiouely on two similar litter and then purchased another 2 pregnant mice 2 weeks later for replicablilty. I repeated for all the groups.

In the papers I am comparing my studies too, most use two-way Anova accounting for sex and treatment. However, how do you compare for similar litters and cage-mates?

Answer 1

There really are two questions here. One has to do with correlations among independent variables, the other with correlations among outcomes.

To deal with correlations arising from things like pups within a single litter, cage-mates, or repeated measurements on the same individual, a mixed model can be a good choice. That can handle nested sets of correlations (e.g., pups within litters within treatment groups) in a straightforward way. Classic Anova works best with a completely balanced design, which you don't seem to have.

A potentially more difficult question is how to evaluate your 4 measures of neurodevelopment together in a way that takes those correlations into account. That would be a truly "multivariate" model in terms of having multiple outcomes. That combination of multiple outcomes with mixed models is outside my expertise, but this page by Ben Bolker on Multivariate analysis with mixed modeling tools in R might provide some guidance. Details will depend on the nature of your outcome measures.