In practice we sometimes used a wisdom that two overlapping CIs for sample mean distribution meant we could not reject Hull hypothesis, to get a quick answer.

It looks though that this is a very raw estimate. I was working on a toy example of a two-sample hypothesis testing when I realized that our "wisdom" was just incorrect.

Am I right?

enter image description here

When I took the difference of samples X and Y, and plotted its moments, apparently the p-value is quite low.

enter image description here

α = 0.01; one-sided test; n(X) = n(Y) = 1000; var(X) = var(Y) = 1.

> t.test(x = xy[sample == 'x', x], y = xy[sample == 'y', x],
+        alternative = "less",
+        mu = 0, paired = F, var.equal = T)

    Two Sample t-test

data:  xy[sample == "x", x] and xy[sample == "y", x]
t = -2.7023, df = 1998, p-value = 0.003472
alternative hypothesis: true difference in means is less than 0
95 percent confidence interval:
        -Inf -0.04665049
sample estimates:
 mean of x  mean of y 
0.02546606 0.14476431 
  • 1
    $\begingroup$ @whuber, thank you, just what I was looking for, sir. $\endgroup$ – Alexey Burnakov Mar 1 '18 at 18:18

Browse other questions tagged or ask your own question.