# Which t-test to use for a two-group pre- post-test design?

I have data on 26 participants (13 from computing and remaining 13 from non computing) who have participated in my research. Each participant is treated with a lab module (Hands on Robotics Session). Now each participant will be evaluated using a rubric on scale of 1 to 4. This experiment has both pre and post test.

For my research i want to evaluate the following questions:

### Research question 1

• Null Hypothesis: students do not learn about computational thinking (programming basics and algorithmic thinking) with the help of robotics.
• Alternate Hypothesis: Students learn about computational thinking (programming basics and algorithmic thinking) with the help of robotics.

To evaluate the above question, the categories i will be considering are Plan, Implementation and Knowledge gained on a scale of Excellent, Good, Fair and Poor.

I think i should use DEPENDENT T-TEST FOR PAIRED SAMPLES. Am i correct?

### Research question 2:

• Null Hypothesis: Participants background (computing and non computing) has no effect in learning about algorithmic thinking with help of robotics
• Alternate Hypothesis:Participants background (computing and non computing) has an effect in learning about algorithmic thinking with help of robotics

I will also evaluate question 2 on a scale of Excellent, Good, Fair and Poor, but with respect to background.

### My Statistical Question

Which T test should I use for each of my research questions?

Since you are interested in measuring an increase pre- and post-test, it seems to me that you should use a paired test.

An issue here is that your variables are likely non Gaussian since they are among 4 categories. If your sample size is big (very roughly larger than 50), then it is no big deal. Otherwise, I would use the Wilcoxon signed rank test which is a non parametric analog of the paired t-test.

• very roughly is right. What if $P( {\rm category \ 1} ) = 1 - 10^{-200}?$ ;) Jun 18, 2012 at 23:40
• Aha (+1), then I am f... But how likely is that? Close to $10^{-200}$ I'd say :-) Jun 18, 2012 at 23:45
• It's also worth noting that this would also require an assumption that the categories comprise an interval scale. It also seems (unless I misread) that, in question 1, the role of the null and alternative are reversed from the way they normally are in a paired $t$-test; the null appears to be that the module does help. Finally, question 2 appears to require a two independent samples test, where the paired differences are the outcome variables. Jun 18, 2012 at 23:51
• I do not subscribe to using the Wilcoxon signed rank test alternative. If you do a simulation study, you can still show the power of the t-test is good --comprable to the signed rank test-- in moderately skewed distributions. If the researchers are interested in testing for a geometric difference in responses, then a log transformation would be justified. Jun 18, 2012 at 23:54
• @marco: Thanks for pointing out that my formation for null and alternate was wrong and thanks for your suggestion on the second question. Jun 19, 2012 at 0:59

gui11aume beat me to it. He said exactly what I had in mind when I read the question. Since scaled measurements are not going to fit a normal error distribution very well a nonparametric paired test is the best way to go in my view and for that I would have recommended the Wilcoxon signed rank test. On the other hand if you were comparing scores by domains where many responses are summed, normality is not a bad assumption and the t test is fairly robust anyway. So in that situation a paired t test might be okay.

I think you are having problems with the nature and elaboration of your alternative hipotesis. Your investigation/reserach hypotesis (and all the theoretical model associated with it) is your alternative hipothesis. Since the variable is clearly ordinal, i´d prefer a Wilcoxon rank test. But anyway i suggest to work out a little bit more your hipothesis.