How to visualize independent two sample t-test?

Question

What are the most accepted ways to visualize the results of an independent two sample t-test? Is a numeric table more often used or some sort of plot? The goal is for a casual observer to look at the figure and immediately see that they are probably from two different populations.

Answer 1

It is worth being clear on the purpose of your plot. In general, there are two different kinds of goals: you can make plots for yourself to assess the assumptions you are making and guide the data analysis process, or you can make plots to communicate a result to others. These are not the same; for example, many viewers / readers of your plot / analysis may be statistically unsophisticated, and may not be familiar with the idea of, say, equal variance and its role in a t-test. You want your plot to convey the important information about your data even to consumers like them. They are implicitly trusting that you have done things correctly. From your question setup, I gather you are after the latter type.

Realistically, the most common and accepted plot for communicating the results of a t-test¹ to others (set aside whether it is actually the most appropriate) is a bar chart of means with standard error bars. This does match the t-test very well in that a t-test compares two means using their standard errors. When you have two independent groups, this will yield a picture that is intuitive, even for the statistically unsophisticated, and (data willing) people can "immediately see that they are probably from two different populations". Here is a simple example using @Tim's data:

nonsmokers <- c(18,22,21,17,20,17,23,20,22,21)
smokers <- c(16,20,14,21,20,18,13,15,17,21)
m        = c(mean(nonsmokers), mean(smokers))
names(m) = c("nonsmokers", "smokers")
se       = c(sd(nonsmokers)/sqrt(length(nonsmokers)), 
             sd(smokers)/sqrt(length(smokers)))
windows()
  bp = barplot(m, ylim=c(16, 21), xpd=FALSE)
  box()
  arrows(x0=bp, y0=m-se, y1=m+se, code=3, angle=90)

That said, data visualization specialists typically disdain these plots. They are often derided as "dynamite plots" (cf., Why dynamite plots are bad). In particular, if you have only a few data, it is often recommended that you simply show the data themselves. If the points overlap, you can jitter them horizontally (add a small amount of random noise) so that they no longer overlap. Because a t-test is fundamentally about means and standard errors, it is best to overlay the means and standard errors onto such a plot. Here is a different version:

set.seed(4643)
plot(jitter(rep(c(0,1), each=10)), c(nonsmokers, smokers), axes=FALSE, 
     xlim=c(-.5, 1.5), xlab="", ylab="")
box()
axis(side=1, at=0:1, labels=c("nonsmokers", "smokers"))
axis(side=2, at=seq(14,22,2))
points(c(0,1), m, pch=15, col="red")
arrows(x0=c(0,1), y0=m-se, y1=m+se, code=3, angle=90, length=.15)

If you have a lot of data, boxplots may be a better choice to get a quick overview of the distributions, and you can overlay the means and SEs there too.

data(randu)
x1 = qnorm(randu[,1])
x2 = qnorm(randu[,2])
m  = c(mean(x1), mean(x2))
se = c(sd(x1)/sqrt(length(x1)), sd(x2)/sqrt(length(x2)))
boxplot(x1, x2)
points(c(1,2), m, pch=15, col="red")
arrows(x0=1:2, y0=m-(1.96*se), y1=m+(1.96*se), code=3, angle=90, length=.1)
# note that I plotted 95% CIs so that they will be easier to see

Simple plots of the data, and boxplots, are sufficiently simple that most people will be able to understand them even if they aren't very statistically savvy. Bear in mind, though, that none of these make it easy to assess the validity of having used a t-test to compare your groups. Those goals are best served by different kinds of plots.

_{1. Note that this discussion assumes an independent samples t-test. These plots could be used with a dependent samples t-test, but could also be misleading in that context (cf., Is using error bars for means in a within-subjects study wrong?).}

Answer 2

The most commonly used way to visualize $t$-test-like comparison is to use boxplots. Below I provide example using dataset describing "relationship between marijuana smoking and a deficit in performance on a task measuring short term memory" from this site.

> nonsmokers <- c(18,22,21,17,20,17,23,20,22,21)
> smokers <- c(16,20,14,21,20,18,13,15,17,21)
> 
> t.test(nonsmokers, smokers)

    Welch Two Sample t-test

data:  nonsmokers and smokers
t = 2.2573, df = 16.376, p-value = 0.03798
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 0.1628205 5.0371795
sample estimates:
mean of x mean of y 
     20.1      17.5 

Actually, boxplots are commonly used for "informal" hypothesis testing, for example as described by Yoav Benjamini in 1988 paper Opening the Box of a Boxplot:

The regular boxplot is supplemented by an approximate confidence interval for the median of the batch, shown as a pair of wedges taken out of the sides of the box. These confidence intervals are constructed in such a way that when two notches of different boxplots do not overlap their medians are significantly different. (...) Since the formula for the confidence interval is a constant times the interquartile range divided by the square root of the batch size, the latter can be perceived from the length of the wedges relative to the length of the box.

See also: T-test using only summary data in a box plot

This plot does not show quantities directly involved in $t$-test, as @NickCox noticed. If you want direct comparison of means with confidence intervals you can use bar plot with marked confidence intervals. Using means and confidence intervals also enables you to conduct hypothesis test (see here or here).

As you can see from other posts and comments under this thread, both boxplots and dynamite plots are somewhat controversial choice, so let me give you one more alternative that was not mentioned yet. First, recall that $t$-test and regression are related. You can plot $t$-test-like comparison as two points with errorbars (confidence intervals) that are connected with line. Slope of the line is proportional to regression slope if you used linear regression rather than $t$-test in this situation. Major advantage of such plot is that it enables you to easily judge the magnitude of difference of means by looking at the slope of the line. It's disadvantage may be that it may suggest that there is some "continuity" between the means (i.e. that you had paired samples).

Boxplots seem to be more commonly used since they provide more information about the distribution of variables visualized (comparing to mean with confidence interval only). They also complement rather than duplicate the information from $t$-test and such usage of plot is encouraged by most style guides, e.g. by Publication Manual of the American Psychological Association:

The first consideration is the information value of the figure in the text of the paper in which it is to appear. If the figure does not add substantively to the understanding of the paper or duplicates other elements of the paper, it should not be included.

Answer 3

This is mostly a variation on the helpful answers by @Tim and @gung, but the graphs cannot be fitted into a comment.

Small but possibly useful points:

1. A strip plot or dot plot as illustrated by @gung needs modification if there are ties, as there are in the example data. Points can be stacked or jittered, or as in the example below you can use a hybrid quantile-box plot as suggested by Emanuel Parzen (most accessible reference is probably 1979. Nonparametric statistical data modeling. Journal, American Statistical Association 74: 105-121). This has other merits too, in underlining that if half the data are inside the box, then half are outside too, and in showing essentially all the detail of the distribution. Where there are just two groups, as there are in this context, any more conventional kind of box plot can be a minimal, indeed skeletal, display. Some would take that as a virtue, but there is scope for showing more detail. The converse argument is that a box plot flagging particular points, notably those more than 1.5 IQR from the nearer quartile, is a clear warning to the user: watch out with a t-test, as there may be points in the tails that you should worry about.

2. You can naturally add an indication of the means to a box plot, which is quite often done. Adding a different marker or point symbol is common. Here we choose reference lines.

Quantile-box plots for smokers and non-smokers. The boxes show medians and quartiles. The horizontal lines in blue show means.

Note. The graph was created in Stata. Here is the code for those interested. stripplot must be installed previously with ssc inst stripplot.

clear 
mat nonsmokers = (18,22,21,17,20,17,23,20,22,21)
mat smokers = (16,20,14,21,20,18,13,15,17,21)
local n = max(colsof(nonsmokers), colsof(smokers)) 
set obs `n' 
gen smokers = smokers[1, _n] 
gen nonsmokers = nonsmokers[1, _n] 
stripplot smokers nonsmokers, vertical cumul centre xla(, noticks) ///
xsc(ra(0.6 2.4)) refline(lcolor(blue)) height(0.5) box ///
ytitle(digit span score) yla(, ang(h)) mcolor(red) msize(medlarge) 

EDIT. This further idea in response to the answer by @Frank Harrell superimposes two normal probability plots (really quantile-quantile plots). The horizontal lines show means. Some would want to add lines for each group indicating perfect fit, e.g. through ($0$, its mean) and ($1$, its mean $+$ its SD) or robust-resistant alternatives.

Answer 4

Besides the nice goal of presenting the results there should be some consideration about which graphics check the assumptions of the two-sample equal variance $t$-test for it to have excellent performance. That would be normal inverse functions of the two empirical cumulative distribution functions. To satisfy the test assumptions these two curves must be parallel straight lines.