Visual display of multiple comparisons test

Question

Suppose, the data below shows the mean response time on a task for respondents among four different groups:

A     B     C    D  
1.2   2.3   4.5  6.7

In order to assess which one of the means are different from one another I do a multiple comparisons test (after an omnibus ANOVA test is cleared) and the multiple comparisons test tells me that the mean for group D is significantly different from the ones for groups A and B and no other pair of differences is significantly different.

What is the best way to present this information visually?

Answer 1

iv <- c("A","B","C","D")
dv <- c(1.2,2.3,4.5,6.7)
gp <- c(1,1,1,2)

par(mai=c(1,1,0,0))
plot(dv, gp, axes=F, xlab="Average time", ylab="Grouping based on 
                   \n mean comparison",
     ylim=c(0,3), xlim=c(0,7), pch=16)
text(dv, gp-.2, iv)
axis(side=2, label=c("i", "ii"), at=c(1,2))
axis(side=1)
abline(h=c(1,2),col="blue",lty=3)

Provide a footnote: Means on the same horizontal reference line are not statistically different from each other. Alpha = 0.05, Bonferroni adjustment

And I really like this design because you can flexibly accomodate group means with multiple memberships. Like in this case, C is not different from D and also not different from A and B:

iv <- c("A","B","C", "C", "D")
dv <- c(1.2,2.3,4.5, 4.5, 6.7)
gp <- c(1,1,1,2,2)

par(mai=c(1,1,0,0))
plot(dv, gp, axes=F, xlab="Average time", ylab="Grouping based on 
            \n mean comparison",
     ylim=c(0,3), xlim=c(0,7), pch=16)
text(dv, gp-.2, iv)
axis(side=2, label=c("i", "ii"), at=c(1,2))
axis(side=1)
abline(h=c(1,2),col="blue",lty=3)

Answer 2

Based on your question and follow-up comments, I'd start with a dot-plot. They're quick and easy (even in Excel). Here's s sample with your data:

This chart type scales well, handles large numbers of data points well and is very easy to understand-even to a non-tech audience.

Answer 3

Point is, that your dataset is too small (4 groups 5 values each). The means obtained from such data are not very accurate representative values for each group - and therefore you should not run ANOVA to make inference about differences among group.

---

One thing is to be understandable to the audience but more important is to be scientifically accurate.

---

I suggest to solve this issue by Kruskal-Wallis followed by multiple comparisons.

Boxplots (with medians) is probably the most used graphical representation of multiple comparisons of groups. To display differences you either make brackets above pairs which are statistically different and add (***-symbols or N.S.) This looks good if you have small number of groups. Or can make notches on each boxplot (very helpful in large number of groups) by which anyone will found desired comparison be eye.

---

You may created boxplots for example in R:

data <- data.frame(value=c(rnorm(60), rnorm(20)+3), 
    group=rep(c("A", "B", "C", "D"), each=20))

              value group
    1  -1.206926025     A
    2  -0.311125313     A
    3   1.336579675     A
    ......
    21  1.543827796     B
    22 -1.874257866     B
    ......
    80  4.383037868     D
    etc.  

boxplot(data$value ~ data$group, notch=TRUE,
    col = "red", xlab="group", ylab="value")

Boxplots shows median values instead of mean. I strongly suggest to not display ONLY mean values for each group. Raw data are the last possibility.