I'd be glad if some of you could help me with my problem. I don't really know, what kind of analysis is the best approach for my Study. I have:

  • 60 Patricipants, 1 Treatment (n=30) and one Controlgroup.
  • The groups are independent
  • The dependent variable is being measured at to times for each group. 1 before treatment or control-treatment and after.
  • The participant are not going to switch groups

Example: Influence of exercise (IV) on mental health (DV). DV of Group 1 is being measured before and after exercise. DV of Group 2 is being measured before and after a Seminar (no exercise).

Basically I want to analyse the effect of exercise on the DV (before and after treatment) and compare it also to the control group. I want to see, if the starting conditions are equal in Group 1 and Group 2 (before treatment) and want to compare the relative change at the second timepoint.

I don't now, wether I have to do a two factorial ANOVA, or a t-test (because I have 2 groups). Couldn't I also do a paired t-test for the treatment group and the controlgroup separately, and a unpaired t test for both groups for the timepoints?

Thank you!


1 Answer 1


If subjects were randomly assigned to the two groups, and measurements are nearly normal, then a Welch 2-sample t test on After-minus-Before differences in measurements should answer the question whether Treatment and Control yield different changes in measurements.

Consider fictitious normal data below, sampled using R.

d.t = round(rnorm(30, 5, 3), 1)
sort(d.t); mean(d.t)
 [1] -3.7  0.7  1.5  1.8  2.1  2.1  2.3  2.4  3.0  3.4  4.0  4.0  4.3  4.4  4.8
[16]  4.8  5.3  5.7  5.8  6.0  6.1  6.2  7.0  7.2  7.6  7.7  8.0  8.1  8.3  8.6
[1] 4.65
d.c = round(rnorm(30, 2, 3), 1)
sort(d.c); mean(d.c)
 [1] -2.7 -2.3 -1.9 -1.4 -0.5 -0.4  0.6  1.2  1.2  1.3  1.5  2.1  2.2  2.4  2.4
[16]  2.5  2.6  2.9  3.0  3.1  3.2  3.9  4.4  4.5  5.0  5.2  5.2  5.5  5.7  7.1
[1] 2.316667

The average difference in measures was greater for the treatment group. The question is whether respective 4.65 is enough larger than 2.32 to be considered statistically significant at some level such as 5%.

That the notches in the the sides of the boxes (treatment on bottom) do not overlap is a preliminary indication of a significant difference.

boxplot(d.t, d.c, horizontal=T, col="skyblue2", notch=T)

enter image description here

There is no indication of skewness or outliers in either boxplot, and Shapiro-Wilk tests do not reject normality. (Both P-values larger than 5%.)

[1] 0.102046
[1] 0.5645659

A Welch two-sample t test (which does not assume equal variances in treatment and control populations) has a very small P-value, so the difference in differences is significant.

t.test(d.t, d.c)

        Welch Two Sample t-test

data:  d.t and d.c
t = 3.414, df = 57.423, p-value = 0.00118
alternative hypothesis: 
 true difference in means is not equal to 0
95 percent confidence interval:
 0.9649392 3.7017275
sample estimates:
mean of x mean of y 
 4.650000  2.316667

Notes:(1) If subjects were not randomized into the two groups, you may not be able to conclude anything useful from your data. It might be useful to look at a two-factor ANOVA for a difference-in-differences design. It is possible that one of the links in the margin denoted as Relevant might be useful. Perhaps more productively, search this site and the Internet of 'difference in differences'.

(2) If data are are clearly not normal and samples are of similar shapes, then a nonparametric Wilcoxon rank sum test may be useful. But that would work best with larger samples. For my fictitious data with $n_1=n_2=30,$ an exact P-value is not available, using R:

wilcox.test(d.t, d.c)$p.val
[1] 0.001514116
Warning message:
In wilcox.test.default(d.t, d.c) : 
 cannot compute exact p-value with ties

(3) It is not clear why attending a lecture is considered a 'control' activity. Couldn't some types of lectures affect mental health?

If subjects were not assigned to groups at random, you might have a significant difference between average Before measurements in the two groups.

Some such issues might be investigated using an ANOVA.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.