I have a case in which I have two sets of ranks. To compare these two ranks, I use once wilcox test and once ks test. The reason why I am using both returns to the special cases of shapes of data which cannot be detected by using only one of these tests. At the moment, I am getting the minimum p-value from these two tests and it works okay. However, I know that this is not correct way. Also, since the p-values come from dependent and different tests p-value combination methods also wont work here.In the results I see two reported statistics as well. Should I use those? how can I get single value (statistic and p-value) from these two tests? Any suggestion would be appreciated.
2
-
$\begingroup$ How you should combine them* depends on what you want to detect, precisely, which should be given by a single, clear alternative hypothesis. $\quad$ * .. where combine two very different statistics makes sense at all; it's not at all clear that it does in this instance. $\endgroup$– Glen_bCommented Mar 2, 2023 at 5:28
-
$\begingroup$ @Glen_b You can have a look into link. I need to detect both differences in my data meaning both are equally important. So I am using two tests at the same time. $\endgroup$– NmghCommented Mar 2, 2023 at 5:45
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
The simplest approach, but also most conservative, is to use a Bonferroni correction. In your case, this would be simply using $2 \times \text{min}(p_1, p_2)$, where $p_1$ and $p_2$ are the p-values from each test.
Note that this is highly conservative in this case: the type 1 error will be exactly equal to the significance only in the case when each test is disjoint (i.e. test 1 results in rejected only if test 2 does not and vice versa). In your case, the your tests are highly positively correlated (i.e. if test 1 results in rejected, it is more likely you will reject in test 2).
