# Can I convert a frequentist p-value into a Bayesian Posterior Probability

I am drafting a teaching session for fellow clinicians to try and provide a somewhat intuitive understanding on how Bayesian statistics differs from the frequentist methods we are taught at medical school. In the session I propose an implausible intervention that is studied and happens to find a difference in sample means with a p-value of 0.048. We then go through comparing the 'oddness' of the results with the 'oddness' of the implausible intervention working. I describe Bayes method of comparing these oddities to try and find which is the least implausible. I turn to this calculator because it uses more concrete inputs than other formulations that need more abstract inputs such as p(data). I then go on to fiddle around with some of the inputs to help build an intuitive sense of what very low p-values achieve.

In my example I use it as follows:

• Calculate the probability of: H0
• Based on the probability of: p-value
• P(H0): 0.999 (I use an average of the class's priors)
• P(p-value|H0): 0.048 (this is the outcome p-value)
• P(p-value|¬H0): 0.8 (I have assumed this is the same as study power)

Questions:

1. Is P(p-value|¬H0) the same as a study power? Or is it only it only the same as the study power if the p-value = the "cut-off p-value". I.e, it varies with the observed p-value?
2. If not, what is it and how could I estimate it and explain it?
3. Is my methodology valid?
• "Testing a Point Null Hypothesis: The Irreconcilability of P Values and Evidence", James O. Berger and Thomas Sellke, Journal of the American Statistical Association, Vol. 82, No. 397 might be relevant here Dec 29, 2023 at 14:18
• The notation is confusing/incorrect. The p-value should be something like p=Pr{data|H0} where you'd need to define "Pr", "data" and "H0" carefully. (Pr is long-run relative frequency, data is statistic as observed or more extreme, H0 means the null is true and test assumptions hold). Also the power is computed under a specific alternative hypothesis. You can't leave it at "not H0". Dec 29, 2023 at 15:15
• That being said, it may be interesting to read this blog by Daniel Lakens: The relation between p-values and the probability H0 is true is not weak enough to ban p-values and references therein. Note however that the post ends with "If you really want to make statements about the probability the null-hypothesis is true, given the data, p-values are not the tool of choice (Bayesian statistics is)." Dec 29, 2023 at 15:21
• Your project seems doomed to fail for the following reasons. Testing a null addresses the question: Are the data reasonably consistent with the null hypothesis? A p-value is a measure of this consistency, and can be regarded as a measure of the absolute plausibility of the null. A different question is: Given the data and two competing hypotheses, what is the relative plausibility of each? Posterior probabilities are Bayesian answers for the relative plausibilty of question. Dec 29, 2023 at 20:15
• I think a great session for the clinicians would be to get them to write down their own prior for the example you have, then ask say 5 of them what their priors are, input those priors one at a time into the Bayes calculator to get 5 posteriors, and report the 5 Bayesian answers back to the class. Then have a discussion about the pros and cons of both methods. Dec 30, 2023 at 18:46

$$P(H_0|\text{data}) = \frac{P(\text{data}|H_0)}{P(\text{data})} \cdot P(H_0)$$
as mentioned in the comments, this $$P(\text{data}|H_0)$$ is not the same as a p-value.