2
$\begingroup$

I have conducted a study with 4 conditions (one control, and three different treatments). Let's label them c1, t1, t2, t3.

Let's assume c1 is significantly different from t1 and t2. I would like to know whether the mean difference between c1 and t1 is bigger than the mean difference between c1 and t2.

Is there a formal statistical test to assess whether the differences are statistically significant?

Alternatively, would it sound logical that the difference between c1 and t2 is bigger than the difference between c1 and t2 if I showed that:

  1. c1 < t1 - significant
  2. c1 < t2 - significant
  3. t1 < t2 - significant
$\endgroup$
3
  • $\begingroup$ I think you have a typo just before your 1-2-3, but I get the point. Why not form d1 = t1-c1 and d2 = t2-c1 and do a t test to see if d2 > d1 is signif. But do the ANOVA first to make sure there are some signif differences before you do other tests. And then use some method (such as Bonferroni) to guard against 'false discovery'. $\endgroup$
    – BruceET
    Jul 31, 2019 at 8:37
  • 1
    $\begingroup$ Fit a linear model. Set c1 as reference. Test the null hypothesis t1=t2 or t1-t2=0. $\endgroup$
    – user158565
    Jul 31, 2019 at 14:10
  • 3
    $\begingroup$ $(\mu_x - \mu_{y_1}) - (\mu_x - \mu_{y_2}) = \mu_{y_2} - \mu_{y_1}$ (and similarly for the sample means). You're simply comparing $y_1$ and $y_2$; $x_1$ doesn't enter into it. Indeed if you properly account for the dependence between $\bar{x}-\bar{y_1}$ and $\bar{x}-\bar{y_2}$, you should end up with the same test as just testing for a difference in the two y-means. $\endgroup$
    – Glen_b
    Aug 1, 2019 at 11:47

1 Answer 1

0
$\begingroup$

With repeated tests the probability of committing a type 1 error increases. This is because, by definition, we reject a null hypothesis if there is less than a 5% chance of obtaining that result assuming the null hypothesis were true. By making 3 comparisons to a control group, we have 3 chances to obtain an unusual result. For this reason, we perform ANOVA first to see if ANY group means are different. If we reject the null hypothesis that means that one or more of the means are different, but we do not know which one is different. Next, we would perform a post-hoc analysis consisting of pairwise t-tests as you have suggested.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.