# How to prove statistical significance, python

I'm new in statistics and would be so grateful if you give me some insights:

I have two big tables - results of work Model_1 and Model_2. I created and calculated statistics - something like Precision = True Positives/(True Positives + False Positives) and suppose that the first model is better that the second one because the proportion in Model_1 is better that in Model_2.

How to prove it statistically? I have an idea to use bootstrap (or just random samples from my two initial samples), calculate the metric there over and over again and see if the distribution of that metric is normal. Is it fine?

Then, if normality is proved it possible to use T-Test (or Mann-Whitney if not) not for mean but for a custom proportion? Theoretically yes, but reading the manual for standard functions:

scipy.stats.ttest_ind(...)[source] Calculate the T-test for the means of two independent samples of scores.

seems like it is all about MEANs.

• If you're able to use SciPy, it has both the T-Test and Mann-Whitney. docs.scipy.org/doc/scipy/reference/generated/… and docs.scipy.org/doc/scipy/reference/generated/… – Matthew0898 Apr 11 at 15:16
• yes, thanks, but I guess before using T-Test or MW, I need to check the normality of distribution. Of which distribution? I can, as I wrote, generate more samples from my initial two (they are tables of 200K and 60K rows), so I can take randomly 1000 rows from each of them, calculate the metric and see if the distribution is normal. That is all that comes in my head - and I wonder is it correct. And then, T-Test, is it applied to a custom metric, NOT the mean-value? – TRP Apr 11 at 15:23
• My apologies, I tried to edit the comment before migration. Perhaps scipy.stats.ttest_ind_from_stats is what you're looking for. – Matthew0898 Apr 11 at 15:26
• Thanks! Yes, that is is really looks like my case. The last thing - how could I get the std. deviations? I have a ratio in first sample, the ratio in second sample. That's it. – TRP Apr 11 at 15:41