Is normality assessed in each individual group or for the entire dataset in repeated measures ANOVA?

Question

This is probably a stupid question, but somehow I got very confused and so far I could not find a clear answer (there might be no clear answer). I have residue data in 4 different plant matrices (eg leaves, flowers, stems), which were obtained from 2 different treatments. Samples were taken on 5 subsequent days from the same plants (repeated measures) (each plant had 5 replicates).

All I wanted to do is to check whether the assumptions for a repeated measures anova or linear mixed model were met, so for example check for normality and equal variances. Assuming I do a Shapiro Wilko test or plot some qq plots: Should I test the entire dataset together? Or should I test each plant matrix individually? Or each treatment? Or each sampling day? Or should I test for each matrix in each treatment on each day? That would leave me with 5 datapoints for a qqplot or test, which seems a bit odd. The same questions is for testing equality of variance (e.g. Levene's test).

Thank you very much for any help to reduce my confusion!

Answer 1

The normality that matters most in regressions is normality of residuals, as standard tests of significance are derived based on that assumption. Even then its importance can be overstated, as some deviations from normality aren't large enough to matter. See this page for extensive discussion about normality. As you sense, normality tests with just a few data points aren't going to be very reliable.

One approach is to start with the overall plot of normality of residuals, which include all of your data points. If there are big problems with your model they might be expected to show up there. Also examine other standard quality-control plots for regressions, scale-location plots and plots of of residuals versus fitted values, versus leverage. If there are problems that are likely to invalidate your modeling approach, they should show up in those plots without your having to do strict normality testing.