1
$\begingroup$

I am trying to follow the work of a scientist who compared two factors (A and B) each with two levels. The general output from R appears as follows:

Estimate Std. Error z value Pr(>|z|)
A.1 B.1 - A.1 B.2 == 0 0.08015 0.05255 1.525 0.42232
A.2 B.2 - A.1 B.2 == 0 0.17510 0.05255 3.332 0.00481 ** 
A.2 B.1 - A.1 B.2 == 0 0.27127 0.05259 5.158 < 0.001 *** 
A.2 B.2 - A.1 B.1 == 0 0.09496 0.05163 1.839 0.25493
A.2 B.1 - A.1 B.1 == 0 0.19112 0.05171 3.696 0.00133 ** 
A.2 B.1 - A.2 B.2 == 0 0.09617 0.05171 1.860 0.24556 

They also have a barchart with the four treatments and a Tukey letter over each one as follows:

A.1 and B.1 (Tukey letter = A,B)
A.1 and B.2 (Tukey letter = A)
A.2 and B.1 (Tukey letter = C)
A.2 and B.2 (Tukey letter = B)

I am a bit confused on how to interpret these two results, as I wonder if they seem conflicting? Below are two issues I am particularly confused about:

1) In the comparison between treatments A.2,B.1 and A.2,B.2, there are two separate Tukey letters (B and C), which I thought would mean these two treatments are statistically different. However, in the table, there is only a P-value of 0.24556 between these two groups, which indicates they are not statistically different. How should I interpret the difference between these two treatment groups?

2) In the comparison between treatments A.1,B.1 and A.1,B.2, the table indicates they have a P-Value of 0.42232, which indicates they are not statistically different. At the same time, their Tukey letters have some overlap but some non-overlap (A and A,B). The fact that both these Tukey letters have "A" means they are not statistically different; however, does the fact that only Tukey letter has "B" mean there is some intermediate statistical difference?

I do not have reproducible R code that made this plot. But it seems the syntax was somewhat as follows:

dat2 = lme(Results ~ Treatment, data=dat, random =~1|Experiment)
anova(dat2)
summary(glht(dat2, linfct=mcp(Treatment="Tukey")))

Thank you for sharing any advice!

$\endgroup$
  • $\begingroup$ Pr(>|z|) does not indicate whether there is significant (or insignificant) difference in effects produced by two groups. $\endgroup$ – Subhash C. Davar Apr 10 '18 at 13:20
1
$\begingroup$

Your concern in 1) is correct. The p-value results do not match those in the compact letter display. Without having the original data and code, it's not known how this happened. It's possible that the compact letter display was created by hand, and an error was introduced. It's also possible that they represent two different post-hoc analyses that had somewhat different results.

Your thinking in 2) is likely to lead you astray. The correct interpretation of a compact letter display is "groups sharing a letter are not significantly different." That's all. It's just a mechanical procedure to go from a table of p-values to the compact letter display.

By the way, the "tukey" in glht doesn't indicate Tukey HSD or Tukey-adjusted comparisons. It just means "do all pairwise comparisons". I believe a single-step adjustment of p-values is applied by default.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.