Paired t-test and Tukey post hoc

Question

I am running a repeated measures anova. Once I get significant p values, I run paired t-tests and I also apply Tukey post hoc.

The fact is that I get lower p values with Tukey than with paired t-tests. How can it occur, if Tukey is more restrictive than paired t-tests?

Thank you very much

---

I first started with paired t-test with bonferroni correction, and then I decided to do the Tukey posthoc test because it was less restrictive than Bonferroni correction. Then is when I saw that results were not the ones I expected.

I run paired t-tests this way between the results in all treatment conditions:

t.test(X1,X2,paired=TRUE) #being x1 the punctuation after treatment 1 and x2 the punctuation after treatment 2.

My results were the following:

Paired t-test

data: 1 and 2 t = 3.0716, df = 23, p-value = 0.005398

data: 1 and 3 t = -0.7863, df = 23, p-value = 0.4397

data: 1 and 4 t = 2.9694, df = 23, p-value = 0.006866

data: 2 and 3 t = -4.5115, df = 23, p-value = 0.0001573

data: 2 and 4 t = -0.79563, df = 23, p-value = 0.4344

data: 3 and 4 t = 3.6949, df = 23, p-value = 0.001196

And I applied Tukey posthoc test this way:

require(nlme) a1<-lme(dep.var.~factortmnt,random=~1|factorid,data=mydata) require(multcomp) summary(glht(a1,linfct=mcp(factortmnt="Tukey")))

My results after Tukey in one of the tests was:

Linear Hypotheses: Estimate Std. Error z value Pr(>|z|)

2 - 1 == 0 -2.5417 0.7051 -3.605 0.00179 **

3 - 1 == 0 0.5417 0.7051 0.768 0.86879

4 - 1 == 0 -2.0417 0.7051 -2.896 0.01965 *

3 - 2 == 0 3.0833 0.7051 4.373 < 0.001 ***

4 - 2 == 0 0.5000 0.7051 0.709 0.89352

4 - 3 == 0 -2.5833 0.7051 -3.664 0.00143 **

I have also tried this code for lme, but I obtain nearly the same results: a1 <- lme(dep.var ~ factortmnt, random = ~ 1|factorid, data = my_data, method = "ML", correlation = corCompSymm(form = ~ factortmnt|factorid))

As you can see, I obtain a lower p value with Tukey than in paired t-test in comparison 1-2, and that's the why I don't understand if Tukey is supposed to be more restrictive than paired t test.

Answer 1

You say that the P values are smaller with the Tukey test, but that is not accurate. In fact, only some of the P values are smaller.

The explanation is that the results are based on different models. The Tukey results are based on a model having an underlying assumption that the error variance is the same for every pairwise comparison. The paired t tests are not based on that model, and each paired test is based on a different subset of the data whereas the original model is fitted to all of the data.