I'm new in statistics and would be so grateful if you give me some insights:
I have two big tables - results of work Model_1 and Model_2. I created and calculated statistics - something like Precision = True Positives/(True Positives + False Positives) and suppose that the first model is better that the second one because the proportion in Model_1 is better that in Model_2.
How to prove it statistically? I have an idea to use bootstrap (or just random samples from my two initial samples), calculate the metric there over and over again and see if the distribution of that metric is normal. Is it fine?
Then, if normality is proved it possible to use T-Test (or Mann-Whitney if not) not for mean but for a custom proportion? Theoretically yes, but reading the manual for standard functions:
scipy.stats.ttest_ind(...)[source] Calculate the T-test for the means of two independent samples of scores.
seems like it is all about MEANs.