Two Samples Test for Discrete Variable and Different Size Groups?

Question

Hello there and apologies for my English and lack of statistic skills!

Two different groups of participants, one comprised of children and the other of adults, run once, at different times, the same test with results in the range of 0 to 60 (i.e. discrete values). The number of participants in each group is different but both are larger than 30. The test is a test of Computational Thinking abilities (think IQ), so I think we can safely assume that the distribution is normal.

I have the Min, Max, Avg, and Std Deviation of the results for both groups. I want to know whether adults do significantly better than kids (my bet is on the kids, though :) ).

I THINK I should use the two samples t-test EXCEPT that this source here states that the test is for continuous data (even though the example it describes is for discrete test scores!)

So, should I use the two samples t-test or should I use another one, and if so - which?

Help!!! :)

Answer 1

First let's answer the statistics question. You want to compare two groups (children and adults) on how they score on a test. The test scores range from 0 to 60 and while scores are discrete they are also ordinal: a score of 50 is higher than a score of 47. Without making any distributional assumptions (eg. normality of test scores), you can use the Wilcoxon-Mann-Whitney rank sum test, or Wilcoxon test for short, to compare the two groups. The null hypothesis is that children and adults rank the same on the test.

However, I'm wondering what you will learn from this analysis. The issue is not with the test itself but with the interpretation of the null hypothesis. Children and adults will score similarly on the test if a) children and adults have the same capacity for computational reasoning; or b) the Computational Reasoning test is not sensitive/precise at measuring this capability. Statistics cannot tell you whether a) or b) holds. And there is also the possibility that the power is low because the sample sizes are relatively small.

A statistical test has two outcomes: reject or not reject the null hypothesis. Let's summarise what we learn with either outcome.

• Null hypothesis is rejected: Children and adults are not equally good at computational thinking and no-one is surprised.
• Null hypothesis is not rejected: Children and adults are equally good at computational thinking, or Computational Thinking abilities is not good at measuring computational thinking, or the power of the statistical test is low.

Separately, there is also the question of how you define "children" vs "adults". Where do you put the break point? In many ways this is an arbitrary choice, so it would be better to use age instead.