# Does a lower pvalue mean that test has higher power?

generic version of question: If you're comparing two statistical tests with different assumptions on the same data, and one gives a lower pvalue than another, does that mean that it has higher power?

biostatistics version: Comparing the SKAT and SKAT-O tests (http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3440237/pdf/kxs014.pdf). SKAT-O is meant to have more power if the genetic variants being tested are unidirectional,so does that mean if it gives a lower (closer to zero) pvalue than SKAT that it does have more power in this scenario and I can assume that the genetic architecture is unidirectional and more like burden tests preferred setup? Im curious because knowing the answer to this would enable me to robustly decide which pvalues to use.

In general, the answer is NO. Suppose you have two different hypothesis tests $T$ and $T'$ for the same hypothesis test problem $H_0$ versus $H_1$ on the same data. Supposedly, $T$ and $T'$ uses different aspects of the data, for example, original data versus ranks. To make a meaningful comparison we must suppose that the two tests have the same significance level $\alpha$, (say =0.05). Or, at least, that is the usual approach.

But, often only p-values are reported without any prior choice of significance level, and the p-value is interpreted as some measure of "strength of evidence". If that is valid, a good measure of strength of evidence (Important: NOT strength of association or effect size!) is of course debated. If going that way, power is not a natural concept, since that depends on the (not choosen!) significance level. The idea, somehow, is that a p-value close to zero is strong evidence against the null hypothesis. That, at least, was Fisher's argument.

How can we now compare the hypothesis tests without the concept of power? We can look at the distribution of $P$ (the p-value). Under the null, for both tests, $P$ is uniformly distributed. We want a test that, under the alternative, tends to give small values of $P$. So now, the two tests can be compared on the basis of the distribution of $P$ under the alternative hypothesis. We want the test which give the $P$ that is "stochastically smaller" in some sense.