I am currently testing with following sample data:
''' Student t test '''
np.random.seed(123) data1 = np.random.randn(1000)*5 +50 #SO: this would have mean 50 and Std dev 5 sns.distplot(data1,hist=True) data2 = np.random.randn(1000)*5 +55 #SO: this would have mean 55 and Std dev 5 sns.distplot(data2,hist=True) np.std(data1) #5.003937687581167 np.std(data2) #4.790047775711126 stat, p = scipy.stats.ttest_ind(data1,data2) print(stat) # = -23.90818904550741 print(p) # = 2.6073235694238827e-111
When I see the plots they look as below.
Question is : P value which the ttest_ind function is giving is well below 0.05. But mean values of data1 and data2 are Not > 5 units apart. ( the Std dev value of both is 5.) AS I am thinking, if Data1 and data2 are > (2*std dev) units apart (approx) then p value should be less than 0.05... ( which means they have indeed statistically significant difference)
I referred to the official documentation of scipy here https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.ttest_ind.html
But did not quite understand using the documentation present there. Am I wrongly interpreting the result? OR : How to understand/Interpret this.?