# How to test a treatment had a positive effect on the experimental group?

For the experiment, 12 people were invited. All participants were pre-tested (10 tasks with 0/1 marks). Then we randomly divided all the participants into two equal groups of 6 people: experimental and control. A treatment was conducted and all participants were post-tested (again 10 task with 0/1 marks).

Results of tests:

data = {'e0':  [6, 5, 5, 5, 6, 0, 0, 0, 1, 0], # experimental group, before a treatment
'c0':  [6, 5, 5, 5, 2, 2, 1, 0, 2, 0], # control group, before a treatment
'e1':  [6, 6, 6, 6, 6, 5, 2, 6, 2, 2], # experimental group, after a treatment
'c1':  [6, 6, 0, 6, 0, 2, 3, 3, 4, 2]} # control group, after a treatment


It is required to test a treatment had a positive effect on the experimental group.

Question. What is a statistical test suitable here?

My attempt is:

$$H_0: \mu_0 = \mu_1$$ vs. $$H_a: \mu_0 < \mu_1,$$ where the $$\mu_i$$ are the respective means for pre- and post-tests in the experimental group.

But in hypothesis statement I don't use the control group results.

• The 10 tasks are obviously not interchangeable: there seems to be 4 hard tasks and 5 easy tasks. Note that simple tasks are not very informative because of a ceiling effect. The 0 in for the third task in the c1 group looks suspicious. The fact that controls do poorly on the fifth task is also interesting but maybe is due to chance because there are only 6 subjects per group. Jun 6, 2022 at 19:59
• Because of this difference among the tasks, it's not clear what the two parameters $\mu_0$ and $\mu_1$ represent. Since your data consists of successes and failures, it's more natural to think in terms of probability of success and to let each task has its own probability of success. Perhaps both the tasks and the subjects can be modeled as random effects. You'd need the full data for this analysis. Jun 6, 2022 at 19:59

You could conduct an independent samples t-test on the post-task scores which would mean that the $$H_0$$ is the control group and $$H_1$$ is the experiment group. However, that does not leverage the baseline scores so if the groups randomly differed at baseline it could lead to faulty conclusions.