# How to visualize independent two sample t-test?

What are the most accepted ways to visualize the results of an independent two sample t-test? Is a numeric table more often used or some sort of plot? The goal is for a casual observer to look at the figure and immediately see that they are probably from two different populations.

• "Accepted" by whom, in what context? "More often used" where? – Glen_b -Reinstate Monica Jan 12 '16 at 1:30

It is worth being clear on the purpose of your plot. In general, there are two different kinds of goals: you can make plots for yourself to assess the assumptions you are making and guide the data analysis process, or you can make plots to communicate a result to others. These are not the same; for example, many viewers / readers of your plot / analysis may be statistically unsophisticated, and may not be familiar with the idea of, say, equal variance and its role in a t-test. You want your plot to convey the important information about your data even to consumers like them. They are implicitly trusting that you have done things correctly. From your question setup, I gather you are after the latter type.

Realistically, the most common and accepted plot for communicating the results of a t-test1 to others (set aside whether it is actually the most appropriate) is a bar chart of means with standard error bars. This does match the t-test very well in that a t-test compares two means using their standard errors. When you have two independent groups, this will yield a picture that is intuitive, even for the statistically unsophisticated, and (data willing) people can "immediately see that they are probably from two different populations". Here is a simple example using @Tim's data:

nonsmokers <- c(18,22,21,17,20,17,23,20,22,21)
smokers <- c(16,20,14,21,20,18,13,15,17,21)
m        = c(mean(nonsmokers), mean(smokers))
names(m) = c("nonsmokers", "smokers")
se       = c(sd(nonsmokers)/sqrt(length(nonsmokers)),
sd(smokers)/sqrt(length(smokers)))
windows()
bp = barplot(m, ylim=c(16, 21), xpd=FALSE)
box()
arrows(x0=bp, y0=m-se, y1=m+se, code=3, angle=90)


That said, data visualization specialists typically disdain these plots. They are often derided as "dynamite plots" (cf., Why dynamite plots are bad). In particular, if you have only a few data, it is often recommended that you simply show the data themselves. If the points overlap, you can jitter them horizontally (add a small amount of random noise) so that they no longer overlap. Because a t-test is fundamentally about means and standard errors, it is best to overlay the means and standard errors onto such a plot. Here is a different version:

set.seed(4643)
plot(jitter(rep(c(0,1), each=10)), c(nonsmokers, smokers), axes=FALSE,
xlim=c(-.5, 1.5), xlab="", ylab="")
box()
axis(side=1, at=0:1, labels=c("nonsmokers", "smokers"))
axis(side=2, at=seq(14,22,2))
points(c(0,1), m, pch=15, col="red")
arrows(x0=c(0,1), y0=m-se, y1=m+se, code=3, angle=90, length=.15)


If you have a lot of data, boxplots may be a better choice to get a quick overview of the distributions, and you can overlay the means and SEs there too.

data(randu)
x1 = qnorm(randu[,1])
x2 = qnorm(randu[,2])
m  = c(mean(x1), mean(x2))
se = c(sd(x1)/sqrt(length(x1)), sd(x2)/sqrt(length(x2)))
boxplot(x1, x2)
points(c(1,2), m, pch=15, col="red")
arrows(x0=1:2, y0=m-(1.96*se), y1=m+(1.96*se), code=3, angle=90, length=.1)
# note that I plotted 95% CIs so that they will be easier to see


Simple plots of the data, and boxplots, are sufficiently simple that most people will be able to understand them even if they aren't very statistically savvy. Bear in mind, though, that none of these make it easy to assess the validity of having used a t-test to compare your groups. Those goals are best served by different kinds of plots.

1. Note that this discussion assumes an independent samples t-test. These plots could be used with a dependent samples t-test, but could also be misleading in that context (cf., Is using error bars for means in a within-subjects study wrong?).

• A big problem with using the bar + CI plot is that sometimes the difference is statistically significant, but the CIs overlap. Given that the bar + CI plot tries to appeal to a generalist audience, we really don't want to spend time explaining this extra wrinkle. – Heisenberg Mar 30 '17 at 16:13
• @Heisenberg, I'm aware of that. There is no single, perfect plot for all occasions and purposes. I also didn't mention CIs, only SEs (although that would be equivalent to some CI, depending on the df). Significance is commonly mentioned in the figure caption, & discussed in the text. If you want it displayed unambiguously in the plot itself, you can add brackets and p-values (eg, here). – gung - Reinstate Monica Mar 30 '17 at 18:07
• My apology if the tone sounds aggressive. I'm just a little miffed that in the end there's still no elegant way to visualize a t-test to a generalist audience without explaining further. It may come down to plotting: 2 bar plots showing group means with SE and a third plot showing the difference and its CI (like your link). But such visualization shows redundant information (groups means AND difference in means), which could also confuse the audience. – Heisenberg Mar 30 '17 at 19:34
• @Heisenberg, I wouldn't use the plot from my other answer unless I were trying to display a within-patients t-test. I think it is probably often fine to present 2 bars w/ SEs for a t-test, w/ a simple figure caption. If you need significance in the plot, you could add brackets or something like that (I generally find it unnecessary). – gung - Reinstate Monica Mar 30 '17 at 19:42

The most commonly used way to visualize $t$-test-like comparison is to use boxplots. Below I provide example using dataset describing "relationship between marijuana smoking and a deficit in performance on a task measuring short term memory" from this site.

> nonsmokers <- c(18,22,21,17,20,17,23,20,22,21)
> smokers <- c(16,20,14,21,20,18,13,15,17,21)
>
> t.test(nonsmokers, smokers)

Welch Two Sample t-test

data:  nonsmokers and smokers
t = 2.2573, df = 16.376, p-value = 0.03798
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
0.1628205 5.0371795
sample estimates:
mean of x mean of y
20.1      17.5


Actually, boxplots are commonly used for "informal" hypothesis testing, for example as described by Yoav Benjamini in 1988 paper Opening the Box of a Boxplot:

The regular boxplot is supplemented by an approximate confidence interval for the median of the batch, shown as a pair of wedges taken out of the sides of the box. These confidence intervals are constructed in such a way that when two notches of different boxplots do not overlap their medians are significantly different. (...) Since the formula for the confidence interval is a constant times the interquartile range divided by the square root of the batch size, the latter can be perceived from the length of the wedges relative to the length of the box.

This plot does not show quantities directly involved in $t$-test, as @NickCox noticed. If you want direct comparison of means with confidence intervals you can use bar plot with marked confidence intervals. Using means and confidence intervals also enables you to conduct hypothesis test (see here or here).

As you can see from other posts and comments under this thread, both boxplots and dynamite plots are somewhat controversial choice, so let me give you one more alternative that was not mentioned yet. First, recall that $t$-test and regression are related. You can plot $t$-test-like comparison as two points with errorbars (confidence intervals) that are connected with line. Slope of the line is proportional to regression slope if you used linear regression rather than $t$-test in this situation. Major advantage of such plot is that it enables you to easily judge the magnitude of difference of means by looking at the slope of the line. It's disadvantage may be that it may suggest that there is some "continuity" between the means (i.e. that you had paired samples).

Boxplots seem to be more commonly used since they provide more information about the distribution of variables visualized (comparing to mean with confidence interval only). They also complement rather than duplicate the information from $t$-test and such usage of plot is encouraged by most style guides, e.g. by Publication Manual of the American Psychological Association:

The first consideration is the information value of the figure in the text of the paper in which it is to appear. If the figure does not add substantively to the understanding of the paper or duplicates other elements of the paper, it should not be included.

• This is indeed common, even in texts that discuss t-tests and ANOVA, but it is an extraordinary choice nevertheless. The box plot doesn't show any of the quantities involved in a t-test directly. Minimally, a pertinent plot should show the means and give more detail on the distribution than does a box plot. The detail within the tails is often crucial in interpreting the test. – Nick Cox Jan 11 '16 at 21:53
• OK @NickCox, the dataset was chosen just for visualization, but now I changed it to more appropriate example. – Tim Jan 11 '16 at 22:06
• The bar plot (aka "dynamite plot") has an extremely high ink:information ratio. – Frank Harrell Jan 12 '16 at 13:15
• The quotation from the APA Manual (I've not checked on the context) is good general advice, but is not in itself a direct argument for using the partially relevant box plot in this context. I agree, naturally, that boxplots can be very useful and often complementary, and that they are very often used, but my first point remains. – Nick Cox Jan 13 '16 at 10:42
• @NickCox I cannot say that I diasagree with you, but I would still consider boxplot to provide additional information and not duplicate t-test results even if it does not directly relates to t-test. It is simple, clear and informative. – Tim Jan 13 '16 at 10:53

This is mostly a variation on the helpful answers by @Tim and @gung, but the graphs cannot be fitted into a comment.

Small but possibly useful points:

1. A strip plot or dot plot as illustrated by @gung needs modification if there are ties, as there are in the example data. Points can be stacked or jittered, or as in the example below you can use a hybrid quantile-box plot as suggested by Emanuel Parzen (most accessible reference is probably 1979. Nonparametric statistical data modeling. Journal, American Statistical Association 74: 105-121). This has other merits too, in underlining that if half the data are inside the box, then half are outside too, and in showing essentially all the detail of the distribution. Where there are just two groups, as there are in this context, any more conventional kind of box plot can be a minimal, indeed skeletal, display. Some would take that as a virtue, but there is scope for showing more detail. The converse argument is that a box plot flagging particular points, notably those more than 1.5 IQR from the nearer quartile, is a clear warning to the user: watch out with a t-test, as there may be points in the tails that you should worry about.

2. You can naturally add an indication of the means to a box plot, which is quite often done. Adding a different marker or point symbol is common. Here we choose reference lines.

Quantile-box plots for smokers and non-smokers. The boxes show medians and quartiles. The horizontal lines in blue show means.

Note. The graph was created in Stata. Here is the code for those interested. stripplot must be installed previously with ssc inst stripplot.

clear
mat nonsmokers = (18,22,21,17,20,17,23,20,22,21)
mat smokers = (16,20,14,21,20,18,13,15,17,21)
local n = max(colsof(nonsmokers), colsof(smokers))
set obs n'
gen smokers = smokers[1, _n]
gen nonsmokers = nonsmokers[1, _n]
stripplot smokers nonsmokers, vertical cumul centre xla(, noticks) ///
xsc(ra(0.6 2.4)) refline(lcolor(blue)) height(0.5) box ///
ytitle(digit span score) yla(, ang(h)) mcolor(red) msize(medlarge)
`

EDIT. This further idea in response to the answer by @Frank Harrell superimposes two normal probability plots (really quantile-quantile plots). The horizontal lines show means. Some would want to add lines for each group indicating perfect fit, e.g. through ($0$, its mean) and ($1$, its mean $+$ its SD) or robust-resistant alternatives.

• +1, I've been meaning to jitter the data, but I haven't been able to get to it yet. I was also thinking of adding more advanced plots, including quantile-box plots, violin plots, & qq-plots, but I ultimately abandoned the idea of listing ever more plots. – gung - Reinstate Monica Jan 12 '16 at 12:41
• @gung Fair enough; unfortunately or fortunately, the thread could easily morph into how to compare two distributions generally. For other data example, histograms could be a serious competitor, etc. – Nick Cox Jan 12 '16 at 13:19

Besides the nice goal of presenting the results there should be some consideration about which graphics check the assumptions of the two-sample equal variance $t$-test for it to have excellent performance. That would be normal inverse functions of the two empirical cumulative distribution functions. To satisfy the test assumptions these two curves must be parallel straight lines.

• I've added one of various possibilities to my own answer. – Nick Cox Jan 12 '16 at 13:55