1
$\begingroup$

I am comparing fluorescence levels in baseline condition, after a pretreatment, and after a treatment with a drug. The statistical software I am using is GraphPad prism.

Using a paired t-test, I have detected a significant difference (p = 0.0382), between my pretreatment group and my treatment group. There is also a highly significant difference between the baseline condition and the pretreatment (p = 0.0001), using a paired t-test, but I am not so interested in this as this effect is well established.

Now as I have three groups, I initially used a repeated measures ANOVA. The ANOVA is highly significant (p <0.0001), which was to be expected due to the large difference between the baseline and pretreatment group. However, the a post-hoc tukey test does not become significant for the pretreatment vs treatment groups, even at P<0.1. So now I am wondering whether it would be justified to indicate a significant difference between the pretreatment and treatment group, given that a t-test is singnificant, but this is not the most appropriate test. Can indicate significance based on the t-test, or do I have to rely on the ANOVA? And what explains the difference between the two?

Plotted data

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Without looking at your data, I can only guess. However it is likely that the t test appears significant because there is no multiplicity penalization. Conversely, the Tukey correction inflates (appropriately) the p value in light of the multiple comparisons. Check attentively the software output and you should be able to recognize this.

Improve this answer
$\endgroup$
2
  • $\begingroup$ That's what I initially thought, and so I re-ran the ANOVA with a bonferroni post-hoc test at a significance level of 0.10, but it still is not significant... But if it is the case that the ANOVA is not significant due to the multiple testing effect, wouldn't it still be fair to state that there is a significant difference between the two groups at a signifance level of P < 0.5? $\endgroup$
    – MaximeH
    Commented Apr 1, 2016 at 13:52
  • $\begingroup$ I all frankness, you should stick to the Tukey inference. If you had planned your study from scratch as a comparison of two groups only, then you could use the t inference unadjusted for multiplicity, but now you are basically forced in terms of frequentist framework to consider such t inference too much at risk of type I error. $\endgroup$
    – Giuseppe Biondi-Zoccai
    Commented Apr 1, 2016 at 19:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.