I cannot understand the results of scipy independent two samples tests on my my dataset. the results of the test as I understand it suggest there is a significant difference between the means of the two populations but the KDE plot shows both curves almost totally overlap both sample groups has ~1000 samples

t test :


result :

Ttest_indResult(statistic=2.224749067750489, pvalue=0.02621349938240159)

KDE plot :

sns.kdeplot(X, bw=.2) sns.kdeplot(Y, bw=.2)

KDE plot of the images

I would expected getting a result with high P-value that expresses the test failure to reject the null hypothesis

  • $\begingroup$ A useful addition to that plot would be color-coded vertical lines at the means of each group. That should show that the Y-value around -1 drags down $\bar{y}$ more than you might think. Combine that with the large sample size, and you've got statistical significance. However, that does not necessarily imply practical significance. You may decide that the difference is too small to matter to your particular problem, and it is okay to do that. $\endgroup$ – Dave Oct 26 '19 at 22:01

The t-test is a test of the difference between two means and KDE plots are not always a good way to look for that. Plus your sample size is pretty big, which makes small difference significant. Plus, although it's hard to tell, it looks like there is an outlier around -1 but only for y.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.