Is there such a thing as a p-value that incorporates a priori power estimations? My understanding is that a p-value of X tells you: if no difference exists, there is an X% chance of observing a difference this extreme.
I haven't exactly thought about how, but it seems like there should be a way to weight that based on the a priori power of the study.
Does that make sense? Does anyone do something like that?
 A: This is an interesting question, and the answer is yes if you also have the prior probability that a hypothesis is true. If you simply construct a contingency table for the null hypothesis using the information from a p-value and an estimate for power, you have the margins for the columns, but you don't have the joint probability or the margins for the rows, and you need the margins for the rows because what you're asking for is the conditional probability for the row where the p-value is significant.
$$ P(A | B) =  \frac {P( A \cap B )} {P(B)} $$
Let A represent the probability of a significant result under the null, and let B represent the probability of a significant result. So A is just the p-value, but to find B, we need to know the prior probability that we are under the null. If the probability that we're under the null is 0.5, we have a p-value of 0.05, and we have power of 0.8, then the probability that we're under the null conditional on a significant result is simply:
$$ .058 =  \frac {0.05 \times 0.5} {(0.8 \times (1 - 0.5)) + (0.05 \times 0.5)} $$
However, if the prior probability that we're under the null is 0.9, then the conditional probability is much smaller.
$$ .36 =  \frac {0.05 \times 0.9} {(0.8 \times (1 - 0.9)) + (0.05 \times 0.9)} $$
And of course if we have less power, the conditional probability is even smaller:
$$ .474 =  \frac {0.05 \times 0.9} {(0.5 \times (1 - 0.9)) + (0.05 \times 0.9)} $$
The problem with all of this is that we usually don't have a good estimate for the prior probability and, as Bruce points out above, we usually don't have a good estimate for power--as such estimates crucially depend on the estimated effect size which is what we want to know in the first place. This is why the approach above is better applied to a field of research than to a single study.
The conditional probability discussed above has been called the false positive report probability (FPRP) and is the compliment of what Ioannidis calls the positive predictive value (PPV). See his widely cited paper "Why Most Published Research Findings are False" where he also considers bias in addition to power and prior probability.
