1
$\begingroup$

I am looking to compare the means of two sample populations of size 10. Ideally I would like to use an independent t-Test however I am concerned that the sample size is too small, as I don't know for sure that they are normally distributed.

The samples are final scores in a video game which was played between subjects under two different conditions (one independent variable).

I can't seem to find a clear answer on what test I should use to make this simple comparison - any advice would be really appreciated.

$\endgroup$

1 Answer 1

1
$\begingroup$

It turns out that someone else on StackExchange asked about t-tests and sample sizes, and the summary appears to be that yes, the t-test is valid even in small sample sizes. You merely need to approximately satisfy the t-test's assumptions. So, you could just go ahead and do a t-test. Your power is what it is with a sample of 10 in each arm.

I've heard one professor say that permutation tests are appropriate for inference in small samples. I haven't yet found anyone saying that they are better than t-tests in small samples, but they are basically only practical in smaller samples, and the link below says they can be a good check of a t-test result. In a permutation test, you would (by my reading of the link) assume that the between-group difference is actually zero, then using your data, calculate all possible permutations of the between-group difference. You're basically calculating an exact p-value for the difference - remember that p-values mean that assuming the null hypothesis is true (i.e. that the difference between groups is zero), what's the probability that you'd have seen the between-group difference you got in your sample?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.