Timeline for Comparing two classifier accuracy results for statistical significance with t-test
Current License: CC BY-SA 4.0
11 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Nov 18, 2022 at 4:38 | comment | added | Exploring | @ÉbeIsaac here sample sizes n1 and n2 are the same. So is the Two Proportion Z-Test will be valid? | |
| Nov 18, 2022 at 4:35 | comment | added | Exploring | @ÉbeIsaac how can you apply Two Proportion Z-Test as here the model1 and model2 both executed over the complete inference dataset. | |
| Nov 18, 2022 at 2:31 | comment | added | Exploring | @ÉbeIsaac which formula you applied from here? Test of Hypothesis -Concise Formula Summary | |
| Aug 9, 2020 at 11:31 | comment | added | Data Man | I like your answer. I wonder about the following though. With binary classification, in practice it is often the case that classes are inbalanced (positives being the minority) and so True Positives are of greater value than True Negatives. This may lead to conclusion that correctly classifying the minority class is a stronger signal of the classifier performance. I wonder whether in such scenario it would be still correct to use the test of proportions that you proposed? Because this test makes no distinction between TP and TN. It treats all those cases equally. | |
| Sep 30, 2018 at 11:44 | comment | added | Ébe Isaac | @ShivaTp Indeed. Thanks for pointing the much needed typo correction. Edit confirmed. | |
| Sep 30, 2018 at 11:44 | history | edited | Ébe Isaac | CC BY-SA 4.0 |
added 1 character in body
|
| Sep 30, 2018 at 11:02 | comment | added | Frans Rodenburg | Though I agree that a test for proportions could be used, there is nothing in the original question that suggests a one-sided test is appropriate. Moreover, "we could say with 95% confidence" is a common misinterpretation. See e.g. here: metheval.uni-jena.de/lehre/0405-ws/evaluationuebung/haller.pdf | |
| Sep 30, 2018 at 10:38 | comment | added | Shiva Tp | Shouldn't $\quad\hat p$ be the average of $\hat p_1$ and $\hat p_2$? So the denominator should be 2n in $\quad\hat p= (x_1+x_2)/2n$. | |
| May 20, 2018 at 23:23 | review | Suggested edits | |||
| May 21, 2018 at 0:57 | |||||
| Nov 14, 2017 at 5:31 | history | edited | Ébe Isaac | CC BY-SA 3.0 |
added 366 characters in body
|
| Nov 14, 2017 at 5:25 | history | answered | Ébe Isaac | CC BY-SA 3.0 |