Using Univariate ANOVA with non-normally distributed data

Question

If my data are non-normally distributed and I'm conducting a 2x2 ANOVA, what can I do to correct for this problem so I can report the main effect and interaction output appropriately?

Only one finding is significant (one of the main effects). I've read that bootstrapping cannot be applied to a univariate ANOVA in SPSS...I also tried this out in SPSS myself and could not obtain any bootstrapped output.

More information:

I used a 2 x 2 (Age [6 years, 7 years] × Educational Classification [Group A, Group B]) univariate ANOVA to explore the hypothesis that test performance (percentage of items correct) would increase with age for children in Group B but would not differ significantly across age for students in Group A.

I transformed the percentages into arcsine values to meet the assumption of a continuous dependent variable.

My sample is small, so the sample size for each cell created by the 2 x 2 are as follows: Group A 6-year-olds = 10, Group A 7-year-olds = 13, Group B 6-year-olds = 20, and Group B 7-year-olds = 14.

I know that scores are non-normally distributed across Group A and across Group B due to most students obtaining high scores on the test (and I ran tests of normality and looked at the Q-Q plots). Group A, which is also the smaller group of the 2, has larger variance.

But should I run normality tests for all 4 groups created by the 2 x 2? I found a Main Effect for Educational Classification but not for Age, and there was no interaction.

I just want to know if my findings are valid given the non-normality, or if I should find a way to fix this problem in SPSS (e.g., bootstrapping).

Answer 1

SPSS aside (I can't help you with that, sorry, I haven't used SPSS in decades), it's a relatively simple matter to use bootstrapping in an ANOVA, but before one even tries to do that it's important to consider what is being assumed and whether it makes sense with your variables. So you should be telling us some things about your response (DV).

The first point to make is that in ANOVA the marginal distribution of the response isn't assumed to be normal; it's the conditional distribution. How are you coming to the conclusion that your data are non-normal (how are you identifying the distribution?), and how non-normal are we talking?

The second point is that the importance of normality changes with sample size, yet you don't mention total sample size.

---

In using the bootstrap, you will need to regard some collection of quantities as exchangeable. In a two-way ANOVA this would normally be some form of residual but for residuals to be exchangeable, you need (for example) the variance of and the shape of the distribution not to change with the mean. These considerations would usually rule out applying it to count data, for example.

You do have alternatives; for some kinds of data you might consider applying a GLM to fit an ANOVA-like model -- I believe that is something you can do in SPSS.

---

Edit in response to the additional information in your edit to your question:

• "Percentage of items correct" is a count (number of items correct) divided by a fixed total (number of items). Scaling aside, this is count data of a sort, for which ANOVA would not normally be appropriate, since you won't likely have linearity (because the response is bounded above and below, though this will only affect the size of the interaction in your case), and the equal variance assumption won't hold across different means (variance must vary as a function of the mean, because of the bounds).

• an arcsin square root transformation would help to stabilize the variance, but it will not "make the data continuous" -- it's still just as discrete as before. It may help a little with the skewness, but it might not make much different.

• There are models more suited to 2x2 count data (e.g. binomial GLMs, loglinear models, even chi-square tests) - but your data might not fit the usual models for counts because test questions are rarely of uniform difficulty and - even withing sub-groups - people are rarely of uniform ability. [You might try a binomial model and see if it's plausible. It's possible that a negative binomial model might be able to deal with the possible heterogeneity.]

• If you're contemplating an arcsin square root transformation, it's usually an indication you should have used a GLM.