2
$\begingroup$

I am running a repeated measures anova. Once I get significant p values, I run paired t-tests and I also apply Tukey post hoc.

The fact is that I get lower p values with Tukey than with paired t-tests. How can it occur, if Tukey is more restrictive than paired t-tests?

Thank you very much


I first started with paired t-test with bonferroni correction, and then I decided to do the Tukey posthoc test because it was less restrictive than Bonferroni correction. Then is when I saw that results were not the ones I expected.

I run paired t-tests this way between the results in all treatment conditions:

t.test(X1,X2,paired=TRUE) #being x1 the punctuation after treatment 1 and x2 the punctuation after treatment 2.

My results were the following:

Paired t-test

data: 1 and 2 t = 3.0716, df = 23, p-value = 0.005398

data: 1 and 3 t = -0.7863, df = 23, p-value = 0.4397

data: 1 and 4 t = 2.9694, df = 23, p-value = 0.006866

data: 2 and 3 t = -4.5115, df = 23, p-value = 0.0001573

data: 2 and 4 t = -0.79563, df = 23, p-value = 0.4344

data: 3 and 4 t = 3.6949, df = 23, p-value = 0.001196

And I applied Tukey posthoc test this way:

require(nlme) a1<-lme(dep.var.~factortmnt,random=~1|factorid,data=mydata) require(multcomp) summary(glht(a1,linfct=mcp(factortmnt="Tukey")))

My results after Tukey in one of the tests was:

Linear Hypotheses: Estimate Std. Error z value Pr(>|z|)

2 - 1 == 0 -2.5417 0.7051 -3.605 0.00179 **

3 - 1 == 0 0.5417 0.7051 0.768 0.86879

4 - 1 == 0 -2.0417 0.7051 -2.896 0.01965 *

3 - 2 == 0 3.0833 0.7051 4.373 < 0.001 ***

4 - 2 == 0 0.5000 0.7051 0.709 0.89352

4 - 3 == 0 -2.5833 0.7051 -3.664 0.00143 **

I have also tried this code for lme, but I obtain nearly the same results: a1 <- lme(dep.var ~ factortmnt, random = ~ 1|factorid, data = my_data, method = "ML", correlation = corCompSymm(form = ~ factortmnt|factorid))

As you can see, I obtain a lower p value with Tukey than in paired t-test in comparison 1-2, and that's the why I don't understand if Tukey is supposed to be more restrictive than paired t test.

$\endgroup$
3
  • 1
    $\begingroup$ Why are you running two different sets of post-hoc tests? $\endgroup$ Dec 1, 2017 at 12:30
  • $\begingroup$ please indicate summary results. $\endgroup$
    – user10619
    Dec 1, 2017 at 12:52
  • $\begingroup$ I have edited my post with the explanation. Thank you very much! $\endgroup$
    – user186605
    Dec 1, 2017 at 14:20

1 Answer 1

1
$\begingroup$

You say that the P values are smaller with the Tukey test, but that is not accurate. In fact, only some of the P values are smaller.

The explanation is that the results are based on different models. The Tukey results are based on a model having an underlying assumption that the error variance is the same for every pairwise comparison. The paired t tests are not based on that model, and each paired test is based on a different subset of the data whereas the original model is fitted to all of the data.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.