What statistical tests should be done if I want to check how a teaching intervention affects student achievement? I have a control group and an experimental group, each group is composed of 16 students who were divided into two groups based on their computed average grade in their Mathematics class - there are equal numbers of low, average, and high performing students in each group. I have their pre-test scores and post-test scores and an independent t-test showed that the pre-test scores of the two groups do not significantly differ. From here, how should I proceed to check and compare student achievement? Thanks in advance!

  • $\begingroup$ Some relevant discussion on this general type of design here. The summary is that ANCOVA is better than the T-test because it controls for the pre-test scores as a precision variable, and thus more precisely measures differences in follow-up. Does this answer your question? $\endgroup$ – AdamO Mar 6 '18 at 18:14
  • $\begingroup$ I'm a newbie in statistics and thus, unfortunately, do not deeply understand ANCOVA. If I understand correctly, the ANCOVA is to be used as an alternative to paired t-test in comparing gain scores within each group. Can it be used to compare the performance of the control and experimental groups (control vs. experimental, post test and gain scores) as well? $\endgroup$ – Teacher Researcher Mar 6 '18 at 18:32
  • $\begingroup$ I think you're on the right track. The idea is to focus on a single comparison. The ANCOVA approach is 1) use only post-scores as an outcome 2) adjust for pre-test scores as a covariate 3) using the intervention assignment as the main effect, and inspect the 95% CI and p-value for a global test of effectiveness of the intervention. The interpretation of the main effect is an expected difference in improvements in the intervention group compared to the control group. $\endgroup$ – AdamO Mar 6 '18 at 18:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.