I have 2 data sets that are applicable to Fisher's method. One has a p-value of 0.07 and the other 0.15. Neither sets were normal or homoscedastic. The first satisfied the ANOVA assumptions only with an arcsine transformation, while the second only with a log transformation. Is it possible to combine the p-values given that their transformations were different?

  • $\begingroup$ The arcsine transformation (presumably, arcsine square root or angular) is designed to stabilise the variance of a proportion. It might also make a distribution closer to normal. A log transformation is what it is. My concern is not whether you can combine P-values here but entirely upstream of that: in what sense do you wish to "combine" data transformed quite differently. If, for example, you had an analysis following on angular for one group and log for another then that could be utterly meaningless and/or incorrect. So, tell us what you are doing precisely. $\endgroup$ – Nick Cox Feb 12 '16 at 12:15

The null hypothesis tested by Fisher's method is that all of the null hypotheses underlying the p-values are true (against the alternative that at least one of those null hypotheses is false). So, as far as the method is concerned, one could combine p-values that test completely different phenomena (e.g., whether the correlation is 0 in this sample, whether the mean is equal to 100 in that sample, and so on). So, you can certainly use Fisher's method to test the 'joint' null hypothesis of any number of tests, whether they bear on the same phenomenon or not.

Now I suspect that in your case you are basically testing the same thing twice, but different transformations were needed to fulfill model assumptions. Again, that's fine as far as the method is concerned.

Some people may object to this, because they consider Fisher's method as a way of pooling information about the underlying effects from multiple studies and if different transformations are used, then this also changes the nature of the underlying effects and then one cannot combine them. But that's a misunderstanding of what the method is actually testing.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.