# How do you estimate the minimum difference between 2 means that pass a t-test?

I am measuring the difference in height between 2 populations. Using the t-test, I can reject the null hypothesis that true means of the 2 populations are not different.

Based on my understanding, this test simply proves that the means are different, but doesn't estimate how different they tend to be. Is this a correct interpretation?

If so, is there a test we can use to estimate this value?

For example, the mean height of population one is 6 feet. The mean eight of population two is 5 feet.

Is there a test I can use that estimates a minimum difference one should expect to see with 95% confidence if you were to pull a different sample out of each populations again at random?

Based on my understanding, this test simply proves that the means are different, but doesn't estimate how different they tend to be.

A) Hypothesis tests prove nothing. They simply say that given the assumption that the two populations means are equal (plus all the other assumptions you make about homogeneity of variance and the distribution of the test statistic), it is incredibly improbable that we see a result at least this extreme or more extreme. Improbable things happen all the time. When we reject the null of a test, we essentially say that we prefer to believe we were wrong rather than believe we were lucky. But that is another conversation entirely. To the meat of your question now...

B) The p-value you get from the test does not tell you about the minimum difference, but a confidence interval can.

Let's start with an example in R. I'll simulate two samples with means 5 and 6 and then perform a t test.

set.seed(0)
pop1 = rnorm(25, 6, 0.75)
pop2 = rnorm(25, 5, 0.75)

t.test(pop1, pop2, var.equal = T)

Two Sample t-test

data:  pop1 and pop2
t = 5.2148, df = 48, p-value = 3.864e-06
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
0.6291752 1.4187887
sample estimates:
mean of x mean of y
6.029939  5.005957


The t.test function computes the difference between pop1 and pop2, so the confidence interval shown is for the difference is between pop1 and pop2.

The smallest difference which is consistent with data at the 95% confidence level is a difference of 0.62 because it is the lower end of our confidence interval.