3
$\begingroup$

I have conducted a study with 4 conditions (one control, and three different treatments). Let's label them c1, t1, t2, t3.

Let's assume c1 is significantly different from t1 and t2. I would like to know whether the mean difference between c1 and t1 is bigger than the mean difference between c1 and t2.

Is there a formal statistical test to assess whether the differences are statistically significant?

Alternatively, would it sound logical that the difference between c1 and t2 is bigger than the difference between c1 and t2 if I showed that:

  1. c1 < t1 - significant
  2. c1 < t2 - significant
  3. t1 < t2 - significant
Improve this question
$\endgroup$
5
  • $\begingroup$ I think you have a typo just before your 1-2-3, but I get the point. Why not form d1 = t1-c1 and d2 = t2-c1 and do a t test to see if d2 > d1 is signif. But do the ANOVA first to make sure there are some signif differences before you do other tests. And then use some method (such as Bonferroni) to guard against 'false discovery'. $\endgroup$
    – BruceET
    Commented Jul 31, 2019 at 8:37
  • 3
    $\begingroup$ Fit a linear model. Set c1 as reference. Test the null hypothesis t1=t2 or t1-t2=0. $\endgroup$
    – user158565
    Commented Jul 31, 2019 at 14:10
  • 3
    $\begingroup$ $(\mu_x - \mu_{y_1}) - (\mu_x - \mu_{y_2}) = \mu_{y_2} - \mu_{y_1}$ (and similarly for the sample means). You're simply comparing $y_1$ and $y_2$; $x_1$ doesn't enter into it. Indeed if you properly account for the dependence between $\bar{x}-\bar{y_1}$ and $\bar{x}-\bar{y_2}$, you should end up with the same test as just testing for a difference in the two y-means. $\endgroup$
    – Glen_b
    Commented Aug 1, 2019 at 11:47
  • $\begingroup$ @user158565 I think that should be t1 as reference? $\endgroup$
    – Estimate the estimators
    Commented Jul 5 at 18:46
  • $\begingroup$ Are these paired experiments? Then the difference is just $$(x_1-y_1) - (x_1-y_2)= y_2-y_1$$ and you can leave out $x_1$ and just compare $y_1$ and $y_2$. $\endgroup$
    – Sextus Empiricus
    Commented Nov 6 at 10:17

2 Answers 2

Reset to default
0
$\begingroup$

With repeated tests the probability of committing a type 1 error increases. This is because, by definition, we reject a null hypothesis if there is less than a 5% chance of obtaining that result assuming the null hypothesis were true. By making 3 comparisons to a control group, we have 3 chances to obtain an unusual result. For this reason, we perform ANOVA first to see if ANY group means are different. If we reject the null hypothesis that means that one or more of the means are different, but we do not know which one is different. Next, we would perform a post-hoc analysis consisting of pairwise t-tests as you have suggested.

Improve this answer
$\endgroup$
0
$\begingroup$

Are these paired experiments? Then the difference is just

$$(x_1-y_1) - (x_1-y_2) = y_2-y_1$$

and you can leave out $x_1$ and just compare $y_1$ and $y_2$ directly.

If they are not paired, then consider the situation described in the question: Interpretation of interaction (time x intervention) in an intervention study when baseline values are unequal The observed difference between $(x_1-y_1)$ and $(x_1-y_2)$ can be due to differences in the baseline/control $x_1$. To correct for that you can add the variable $x_1$ into the model as a covariate.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.