I have control and intervention groups (N = 50 and 75, respectively) for whom 15 tests (all having quantitative results) were done at baseline and at 3 months.

It is not a randomized study and the baseline values in control group ARE different from those of intervention group.

My precise question is: "Does the intervention causes a significant change in values of these tests"

What is the best method for this? Should I perform unpaired t-tests on baseline-follow up differences in 2 groups or should I use anova/regression?

Also, how do I correct for multiple tests being done here?

If this has already been discussed, please point me to the right link(s). Thanks for your help.

Edit: Data is in following format:

ID_NO   GRP prepost test1   test2
1   active  pre     10      0.074
2   control pre     11      0.053
1   active  post    10.8    0.042
2   control post    10.5    0.039

For anova, following can be used (in R):

summary(aov(testresult ~ GRP * prepost + Error(ID_NO/prepost), data=mydata))

Following can be used for regression:

summary(lm(testresult_difference ~ testresult_basal + GRP , data=mydata))

Unpaired t-tests can be used for testing difference (change) in controls vs change in intervention group. Similarly unpaired t-test can be used for comparing post/pre ratio in controls vs that intervention group.

Which method should I use?

  • $\begingroup$ How many tests were given? If you have a lot of tests you might need to worry about inflated type I errors/p-values, so you would want to consider something like a Bonferroni correction. $\endgroup$ – robin.datadrivers Mar 7 '15 at 19:17
  • $\begingroup$ @robin.datadrivers : About 15 tests have been performed. So I think correction may be needed. How do I integrate that with analysis? I have added this to my question above. $\endgroup$ – rnso Mar 8 '15 at 2:47

You should do an independent T-test between the two groups using the difference in each student's pre- and post- scores as the dependent variable.

  • 2
    $\begingroup$ But note that a similarly constructed regression analysis will give you the same results. $\endgroup$ – StatsStudent Mar 7 '15 at 18:29
  • 1
    $\begingroup$ I'd also add that you may want to try to do a propensity score analysis and adjustment if you are trying to determine causal effect of the treatment on outcome. This will help you to identify covariates where treatment and control do not overlap and could be confounding your results. $\endgroup$ – StatsStudent Jan 8 '16 at 2:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.