Total scores are normally distributed, but subtest scores are not; what to do?

Question

I have two sets of data for males and females: $n=33$. I'd like to compare test results for both groups. I want to do a $t$ test, and I know my data should be normally distributed if I want to use a parametric test. I also know that if $n>30$, then the $t$ test normality assumption can be violated. Anyway, what would you suggest? Test total scores are normally distributed for both groups, but 6 of the subtest scores are not.

Answer 1

Too long for a comment; hopefully it will expand to become a better answer as issues become clarified.

Much of what you say you know ain't so!

I have two sets of data for males and females: n is 33.

Is that 33 for each group or 33 in total?

I want to do a t test,

Why would "doing a t-test" rather than answering the question you want to ask the data (to which 'do a t-test' might be one answer) be what you want?

(It's like saying "I want to drive the car" when you really mean "I want to go buy something from the hardware store". It is possible you might want to drive a car because you think it's a fun activity -- I know I enjoy doing a t-test now and again ... but I bet that's not the case here. )

and I know my data should be normally distributed if I want to use a parametric test.

Not so. If you want to do a parametric test that also assumes normality, then yes, it's good to have something like a normal distribution, especially with small samples (when you have no good chance of telling when it's violated).

I could do something akin to a t-test (a two-sample comparison of means) using a GLM, for example, that assumed some distribution other than normal (say a Gamma, for example), and it would still be a parametric test.

But for some tests that assume normality, and some non-normal distributions it matters a lot less than with others (some parametric tests of equality of variance, for example, are quite sensitive to normality).

I also know that if n is larger than 30, then the t test normality assumption can be violated.

Also not so. Indeed, I think this is bad advice. In some cases you can happily violate normality with $n=2$ (if it's non-normal in ways that mean the resulting t-test still behaves quite reasonably for your purposes). In other cases, $n=1000$ isn't enough -- but in that case sometimes $n=100,000$ might be fine.

The value "30", for all its popularity in certain areas, is an arbitrary number, it has no general applicability.

But any way I want to ask you what you would suggest. Test total scores are normally distributed for both groups, but 6 of the subtest scores are not.

How do you know some quantity is normally distributed? (From the following sentence I get the impression your scores are probably both discrete and bounded and so, a priori not normal.)

The distribution of subtest scores will only matter if you need to compare subtest scores.

(Highlighted, because that sentence responds directly to the question in the title and it might not be noticed if I don't make it obvious.)

---

The first things to consider, in approximate order:

1) What questions do you need to answer? (You have to tell us this one)

2) What's a good way to answer those questions? (This is where we might be of some help)

Some general comments:

• With heavily discrete, moderately skewed data, a Wilcoxon-Mann-Whitney test may not be a substantially better choice than a t-test (indeed, it might be worse).

• testing normality to see if you can use a t-test answers the wrong question (you're interested in how the properties of your test are affected, rather than whether you have a large enough sample to pick up the difference)

• testing normality and then choosing a test conditional on the outcome of that test leads the tests you do when you make the comparison to not have the desired properties. See some of the links given here. (It may be better, for example, to simply not assume normality with very small sample sizes.)

If you can make the necessary assumptions, at least under the null (such as exchangeability), you might consider some form of permutation test, perhaps.