Is the Chi-Square test for association (comparing two proportions) or a Fisher's exact test more powerful than a chi-square goodness of fit test (comparing multiple proportions to determine whether at least one proportion among several is different from the others)?
$\begingroup$
$\endgroup$
1
-
1$\begingroup$ What is the basis on which to compare power for tests of very different hypotheses? You could compare power of tests of the same null against the same alternative (apples with apples), but not for completely different kinds of test. It's a bit like asking who is better out of this novelist and that wrestler. $\endgroup$– Glen_bCommented Aug 19, 2021 at 16:44
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
9
Power always depends on a specific alternative; most tests including these can have very low power against close alternatives (to the $H_0$) and high power against far alternatives. As the tests you are asking to compare here are for different types of problem (and therefore the alternatives to compare power on are also different), powers cannot be compared.
answered Aug 19, 2021 at 15:32
-
$\begingroup$ Yes, but the power is the value obtained from an integral. The integration region differs depending on your alternative hypothesis and critical region, but the power is simply a value, you can always compare floating points. $\endgroup$– user318514Commented Aug 19, 2021 at 15:36
-
1$\begingroup$ You can compare them if you compute power for the same alternative. If the alternatives are not the same, how will you choose alternative 1 for test 1 and alternative 2 for test 2 to compare the power values? $\endgroup$ Commented Aug 19, 2021 at 15:37
-
1$\begingroup$ In response to a question that seems to have been deleted in the meantime: As I wrote, power is computed on alternatives, so in the first place the alternatives have to be the same. However, it will be in most cases hard to argue that a comparison makes sense if the alternative is the same but the null hypothesis is different. (You could compare power against $\mu=1$, say, for a two-sided test with $H_0:\ \mu=0$, and for a one-sided test with $H_0:\ \mu\le 0$, although arguably these null hypotheses are still "pretty much the same".) $\endgroup$ Commented Aug 19, 2021 at 15:44
-
$\begingroup$ but mu=-50 versus mu=100 is not the same by the KL divergence? $\endgroup$– user318514Commented Aug 19, 2021 at 15:45
-
1$\begingroup$ Sorry, I don't even get what KL divergence has to do with this, and the values of mu have no meaning unless you define the problem in detail. (I do realise this applies to my own comment, too, but I hope it makes its point anyway. ;-) $\endgroup$ Commented Aug 19, 2021 at 15:47