# The difference between two population means

I am trying to answer this question:

There is no difference between two population means. From each of these two populations
a sample is selected and then for both samples the means are calculated.

Consider this:
"since the null hypo is true, a significant result can never be found when we perform
a t-test for independent measures (two-tailed, alpha = 0.05)


Can we say that is statement is in/correct, partially correct, or insufficient to answer? I am really unsure of which path to take.

This is totally false. Let's do an R simulation.

set.seed(2021)
ps <- rep(NA, 1000)
for (i in 1:1000){

# Simulate data from two identical distributions
#
x <- rnorm(100)
y <- rnorm(100)

# Test the means, save the p-value
#
ps[i] <- t.test(x, y, var.equal=T)$p.value } hist(ps) summary(ps)  When you run this, you will notice that a number of the p-values fall below $$0.05$$. In fact, about $$5\%$$ will fall below $$0.05$$. It turns out that, under the null hypothesis, the p-value has a uniform distribution on $$[0,1]$$. • You can prove a far stronger and far-reaching statement: no matter what the populations may be, provided only that at least one of them is not constant, and no matter what significance level you choose, the chance of finding a significant difference (using a t test) is always positive. – whuber Jan 14, 2021 at 21:25 • I really appreciate both of your comments and effort in helping me out. I sincerely thank you both. Jan 14, 2021 at 21:34 • @whuber, can I ask what you meant by being not constant for a population? Also, should I understand that even if these two populations not only had the same mean but also the same standard deviation, there is still a positive chance of finding a significant difference? Jan 14, 2021 at 21:53 • @e.erhan The example I gave uses identical distributions: same mean, same standard deviation, same everything.$\text{//}\$ A constant population is a population where every observation has the same value.
– Dave
Jan 14, 2021 at 21:55
• @Dave Thanks for the clarification! Jan 14, 2021 at 21:59