I have a question that's very similar to this question: Combine multiple independent permutation tests
My 'issue' is exactly the same: I have 20 subjects and I used permutation tests to determine the correlation between one measure and another, giving me 20 test-statistics/pvalues that I want to combine in one single p-value. Note that the actual case is a bit more complicated than this, so mixed effects modelling or other methods to analyse the data in one go are not appropriate in this case.
The null hypothesis (I think) I want to test is that the 20 test statistics / pvalues combined are not different from a random uniform distribution. The alternative hypothesis is that the test statistics combined are larger than what can be expected by chance (or that the pvalues are lower than can be expected by chance). What can I do to test this hypothesis?
The permutations I have been doing are not computationally very demanding, so that's not a limiting factor in terms of possible ways to address this.
NOTE: In the answer to the question cited above, it was suggested to use Fisher's method as a way of combining p values, but I'm not sure whether that's the right test to use in this case, since it tests the null hypothesis that all of separate null hypotheses are true. The alternative is that one of the alternative hypotheses are true, and that's to anti-conservative I think.
Thank you for your help!