# Is a paired samples t-test suitable in this case?

I have 3 tasks and want to find out which one is more effective (that is, to find which task people performed better in).

Just over 30 people have participated in all 3 tasks, and their performance gives a numerical score in each.

Would it be appropriate to use a paired samples t test to compare performance between tasks A and B, A and C and B and C? What would be the best way of approaching data analysis in this instance?

Thank you so much.

I would recommend a linear mixed model, the participants representing the random effects. Then you would be able to get all three comparisons at once, with a pooled variance.

• Thanks so much, I'll take a look. Just to confirm, would it still be appropriate to use a paired samples t-test for the instance described though, since I'm a bit more familiar with it? Commented Jun 1, 2022 at 11:56
• @Rafa73 Not quite, for ex. if you perform a comparison between A and B you lose valuable information about C, which you do not account for. A linear mixed model would consider all three when performing the comparisons. Commented Jun 1, 2022 at 12:09

There are two options, which are appropriate in different scenarios:

1. perform t-tests for each pair and use a multiple comparisons adjustment to correct for family-wise error rate

2. perform a random-effects ANOVA test to compare all 3 groups, and if you get a significant finding, do pairwise t-test comparisons using the pooled variance and a multiple comparisons adjustment

In both cases, you end up doing pairwise t-test comparisons. The difference is whether you're using the individual-group variances in those t-tests or using the pooled variance from the ANOVA calculation. #1 would be more appropriate if you have reason to believe that the groups don't have the same variances. #2 would be appropriate if you believe there is a single variance, and information from the groups should be aggregated to estimate the pooled variance.

If you end up taking approach #1, then you'd use the unequal variances t-test (Welch's t-test).

There are other assumptions to satisfy, like normality of the data--or a large enough sample size for the central limit theorem to apply, such that the sample means are normally distributed. If you're doing this in a classroom setting, then n≥30 is a common threshold for assuming that the CLT applies. If you're in a research setting, then it's entirely possible n=30 is far too small; it depends on the underlying distribution. If you have information about that distribution, you can run simulations where you draw random samples from the underlying distribution with n=30, compute the mean for each, and look at the sampling distribution of the mean to evaluate whether it's normal.

There are many ways to correct for multiple comparisons. I think Tukey and Bonferroni's method are the most common, but someone with more experience could weigh in here.

• It's worth mentioning that ANOVA models are mathematically equivalent to linear models with dummy/indicator variables for the levels of a categorical variable. So the random effects ANOVA model I suggest is the same as the random effects linear model that @user2974951 suggests. I stuck with the ANOVA/hypothesis testing framework in my answer, but in many circumstances, I prefer to think about it the way user2974951 mentions, as a linear model. Commented Jun 1, 2022 at 13:33