0
$\begingroup$

Im new here and also consider me as a total beginner in statistics. So in my scenario, lets say there's two learning interventions, Intervention A and B. The participants are divided into two groups, one group receiving A (Group 1), another receive B (Group 2). These two groups undergo pre-test and post-test. So let's say after conducting paired T-test, it is found that both groups A and B has significant difference in their pre and post test scores, in which both groups increased their scores significantly. Since both groups are actually significant i.e the intervention is both effective in increasing the two groups' scores, is there a way to compare whether Intervention A is MORE effective than B or the other way around?

$\endgroup$
1
  • 1
    $\begingroup$ Look at what is called effect size. $\endgroup$
    – usεr11852
    Dec 10, 2023 at 12:02

1 Answer 1

1
$\begingroup$

Your question is an excellent way to highlight a core problem with the significant/not significant approach to data analysis: it obscures the actual result by distilling it into a single dichotomy.

The first thing to do is to examine the data. Not by statistical analysis, but by eye. Make good graphs of the results where the pre and post results for each individual are connected and see if there is an obvious difference in the effect between the groups. Plot the pre-post differences for each participant and see if there is an obvious difference.

Then think about a statistical analysis that directly documents (or tests) that difference. (Yes, ideally you would have thought about the analysis in advance.) Quite likely you will find that an ordinary Student's t-test will suffice, or maybe a non-parametric analog like a permutations test. Do not express the results of that test as simply significant or not significant as that then obscures the evidence that you found in the data. Say what the p-value is and give confidence intervals for the pre-post differences in both groups, and show the data in a graphical form.

If you want to know more about significant/not significant and p-values then I strongly advise you to look at this chapter on the topic: A Reckless Guide to P-values.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.