I have a maybe really silly question about t-test. I know that t-test is a test for comparing means between two sets of data. But I also was taught that when 2 SD cross over with each between the data (ie 2 (mean +/- 1 SD) ranges overlap), it means there is no statistical difference in between two groups. But my result was different than what I was expected. Two mean +/- 1 SD overlap with each other (ex mean1= 28.01, mean2=28.96 and the SDs are about 10 for each mean) but has p-value at 0.003. Can someone tell me how did this happen and what should I pay attention to?

ps. N>30K and the data is heteroscedastic.

• It sounds like the t-test is comparing each mean with zero rather than to each other. – HStamper Apr 21 '17 at 21:51
• (1) You appear to confuse the SD with the SE. (2) Even when the SE ranges overlap, the difference can be significant: see stats.stackexchange.com/a/31660/919 for some remarks and stats.stackexchange.com/a/18259/919 for a quantitative analysis of how much overlap can be considered significant. – whuber Apr 21 '17 at 22:42

Hm... I'm not an expert, but your t-test result doesn't seem all that surprising to me. If we assume that your samples are in fact drawn from different populations (let's say with $\mu_1 = 28.01, \sigma_1 = 10$ and $\mu_2 = 28.96, \sigma_2 = 10$) then it's entirely possible that both

1. their SDs could overlap, and
2. a t-test could correctly detect the difference between the two group means.

Yes, the difference is small, but 30K points is an awful lot of data, and more data makes the t-test more sensitive.

But I also was taught that when 2 SD cross over with each between the data (ie 2 (mean +/- 1 SD) ranges overlap), it means there is no statistical difference in between two groups.

^ Are you sure this is correct?